-
Notifications
You must be signed in to change notification settings - Fork 447
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
[GLUTEN-3378][VL] Iceberg support set partition columns in split info #3987
Conversation
Thanks for opening a pull request! Could you open an issue for this pull request on Github Issues? https://github.com/oap-project/gluten/issues Then could you also rename commit message and pull request title in the following format?
See also: |
Run Gluten Clickhouse CI |
@YannByron @yma11 Could you help review? |
Run Gluten Clickhouse CI |
ff79ef8
to
435e773
Compare
Run Gluten Clickhouse CI |
gluten-iceberg/src/main/scala/org/apache/iceberg/spark/source/GlutenIcebergSourceUtil.scala
Show resolved
Hide resolved
Run Gluten Clickhouse CI |
8bf55a7
to
86348a9
Compare
Run Gluten Clickhouse CI |
86348a9
to
19db4e3
Compare
Run Gluten Clickhouse CI |
gluten-iceberg/src/test/scala/io/glutenproject/execution/VeloxTPCHIcebergSuite.scala
Show resolved
Hide resolved
19db4e3
to
9d5e9b8
Compare
Run Gluten Clickhouse CI |
@liujiayi771 please rebase this PR to resolve conflicts. |
9d5e9b8
to
6006880
Compare
Run Gluten Clickhouse CI |
@yma11 Rebased and CI passed. |
===== Performance report for TPCH SF2000 with Velox backend, for reference only ====
|
What changes were proposed in this pull request?
Velox support specifying
partitionColumns
in the split info, which improves efficiency when reading partition columns as there's no need to read from the parquet file.