Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

feat(parquet): Add Int64 Timestamp in reader #11530

Open
wants to merge 1 commit into
base: main
Choose a base branch
from
Open

feat(parquet): Add Int64 Timestamp in reader #11530

wants to merge 1 commit into from
+176 −15

Conversation

zuyu
Copy link
Contributor

@zuyu zuyu commented Nov 13, 2024 •
edited
Loading

Support for both plain and dictionary encoding.

@facebook-github-bot facebook-github-bot added the CLA Signed This label is managed by the Facebook bot. Authors need to sign the CLA before a PR can be reviewed. label Nov 13, 2024
@netlify Netlify
Copy link

netlify bot commented Nov 13, 2024 •
edited
Loading

✅ Deploy Preview for meta-velox canceled.

Name Link
🔨 Latest commit 0aa7a4d
🔍 Latest deploy log https://app.netlify.com/sites/meta-velox/deploys/675d33f75bc99d00086f9d36

@zuyu zuyu mentioned this pull request Nov 13, 2024
Open
@zuyu zuyu changed the title Add Int64 Timestamp support in Parquet reader <feat>(parquet): Add Int64 Timestamp support in Parquet reader Nov 14, 2024
@zuyu zuyu changed the title <feat>(parquet): Add Int64 Timestamp support in Parquet reader <feat>(parquet): Add Int64 Timestamp in reader Nov 14, 2024
@zuyu zuyu changed the title <feat>(parquet): Add Int64 Timestamp in reader feat(parquet): Add Int64 Timestamp in reader Nov 14, 2024
@liujiayi771
Copy link
Contributor

Could you add some unit tests?

yingsu00
yingsu00 reviewed Nov 18, 2024
View reviewed changes
Copy link
Collaborator

@yingsu00 yingsu00 left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@zuyu Can you please add a commit and PR message describing the change?

surnaik
surnaik suggested changes Nov 20, 2024
View reviewed changes
velox/dwio/common/IntDecoder.h Outdated Show resolved Hide resolved
velox/dwio/common/IntDecoder.h Outdated Show resolved Hide resolved
velox/dwio/parquet/reader/TimestampColumnReader.h Outdated Show resolved Hide resolved
@surnaik
Copy link

surnaik commented Nov 20, 2024

I am assuming you're also planning to add support for Dictionary encoding and range filters, currently the range filter is always casted to INT96 range filter.

@zuyu zuyu mentioned this pull request Nov 20, 2024
Open
@zuyu zuyu force-pushed the ts-int64 branch 3 times, most recently from 2c824cd to 43dc8a9 Compare December 3, 2024 22:13
@zuyu zuyu marked this pull request as ready for review December 11, 2024 19:58
@zuyu zuyu requested a review from majetideepak as a code owner December 11, 2024 19:58
@zuyu zuyu requested review from Yuhta, yingsu00 and surnaik December 11, 2024 19:58
@zuyu zuyu mentioned this pull request Dec 11, 2024
Open
@zuyu zuyu force-pushed the ts-int64 branch 3 times, most recently from 64ab6f2 to c08c59c Compare December 13, 2024 19:44
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Reviewers

@majetideepak majetideepak Awaiting requested review from majetideepak majetideepak is a code owner

@Yuhta Yuhta Awaiting requested review from Yuhta

@yingsu00 yingsu00 Awaiting requested review from yingsu00

@surnaik surnaik Awaiting requested review from surnaik

Assignees
No one assigned
Labels
CLA Signed This label is managed by the Facebook bot. Authors need to sign the CLA before a PR can be reviewed.
Projects
None yet
Milestone
No milestone
Development

Successfully merging this pull request may close these issues.
6 participants
@zuyu @liujiayi771 @surnaik @Yuhta @yingsu00 @facebook-github-bot