DNM

[acvictor]

What changes were proposed in this pull request?

(Please fill in changes proposed in this fix)

(Fixes: #ISSUE-ID)

How was this patch tested?

(Please explain how this patch was tested. E.g. unit tests, integration tests, manual tests)

(If this patch involves UI changes, please attach a screenshot; otherwise, remove this)

[github-actions]

Thanks for opening a pull request!

Could you open an issue for this pull request on Github Issues?

https://github.com/apache/incubator-gluten/issues

Then could you also rename commit message and pull request title in the following format?

[GLUTEN-${ISSUES_ID}][COMPONENT]feat/fix: ${detailed message}

See also:

• Other pull requests

[github-actions]

Run Gluten Clickhouse CI

[FelixYBW]

Can you comment a bit how these info are used by Velox? Is it aligned with Spark's behavior?

[gaoyangxiaozhu]

can't we leverage metadatacolumns for this since it already contains fileSize and modifyTime ?

[acvictor]

can't we leverage metadatacolumns for this since it already contains fileSize and modifyTime ?

Does metadatacolumns always contain file size and modification time? I understood it as containing these only if they are being queried. In this case I want it to always be passed in the split.

[github-actions]

Run Gluten Clickhouse CI

[github-actions]

Run Gluten Clickhouse CI

[github-actions]

Run Gluten Clickhouse CI

[github-actions]

Run Gluten Clickhouse CI

[github-actions]

Run Gluten Clickhouse CI

[acvictor]

Can you comment a bit how these info are used by Velox? Is it aligned with Spark's behavior?

In Spark, properties are present in the split and obtained during listing. Earlier Velox would make one additional call to remote storage to fetch file length per openFileForRead call on a path even though this information is already present upstream wihle constructing the split. Velox now allows these values to be passed from the caller. By making this change we can eliminate one additional call to remote storage per path (RTT is in the order of 10s of milli seconds).

[github-actions]

Run Gluten Clickhouse CI

[github-actions]

Run Gluten Clickhouse CI

[github-actions]

Run Gluten Clickhouse CI

[github-actions]

Run Gluten Clickhouse CI

[github-actions]

Run Gluten Clickhouse CI

[github-actions]

Run Gluten Clickhouse CI

[github-actions]

Run Gluten Clickhouse CI

[github-actions]

Run Gluten Clickhouse CI

[github-actions]

Run Gluten Clickhouse CI

[github-actions]

Run Gluten Clickhouse CI

[github-actions]

This PR is stale because it has been open 45 days with no activity. Remove stale label or comment or this will be closed in 10 days.

[github-actions]

This PR was auto-closed because it has been stalled for 10 days with no activity. Please feel free to reopen if it is still valid. Thanks.