Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[VL] Align the timezone of timestamp partition value with Velox #8015

Draft
wants to merge 1 commit into
base: main
Choose a base branch
from

Conversation

rui-mo
Copy link
Contributor

@rui-mo rui-mo commented Nov 21, 2024

What changes were proposed in this pull request?

In this way we don't need additional change for Velox.

How was this patch tested?

under test

Copy link

Thanks for opening a pull request!

Could you open an issue for this pull request on Github Issues?

https://github.com/apache/incubator-gluten/issues

Then could you also rename commit message and pull request title in the following format?

[GLUTEN-${ISSUES_ID}][COMPONENT]feat/fix: ${detailed message}

See also:

@surnaik
Copy link
Contributor

surnaik commented Nov 26, 2024

Hey @rui-mo , when we construct Velox Timestamp object from either INT96 or INT64, is the timezone by default "America/Los Angeles"? - https://github.com/facebookincubator/velox/blob/c395c553bd9627b4a3211e34f16bb1949bbcd43a/velox/dwio/parquet/reader/TimestampColumnReader.h#L95

If the default timezone here is "America/Los Angeles", should we offset it assuming the underlying timezone for the timestamp is UTC?

@rui-mo
Copy link
Contributor Author

rui-mo commented Nov 26, 2024

Hi @surnaik, this issue is about the timestamp partition value, which is converted from string in Velox. During this conversion, the default timezone (which is set as "America/Los Angeles" in Velox) is used. This PR tries to adopt the same timezone as Velox to see if we can remove this additional change: oap-project/velox@567f7df.

https://github.com/facebookincubator/velox/blob/c395c553bd9627b4a3211e34f16bb1949bbcd43a/velox/connectors/hive/SplitReader.cpp#L58-L60

https://github.com/facebookincubator/velox/blob/c395c553bd9627b4a3211e34f16bb1949bbcd43a/velox/type/Timestamp.cpp#L96-L101

@rui-mo rui-mo changed the title [VL] Align the timezone of decimal partition value with Velox [VL] Align the timezone of timestamp partition value with Velox Dec 23, 2024
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

Successfully merging this pull request may close these issues.

2 participants