Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Parquet Sink for substreams #132

Merged
merged 48 commits into from
Aug 14, 2023
Merged

Parquet Sink for substreams #132

merged 48 commits into from
Aug 14, 2023

Conversation

robinbernon
Copy link
Contributor

No description provided.

@robinbernon robinbernon marked this pull request as draft February 27, 2023 02:01
jaimehgb
jaimehgb reviewed Mar 7, 2023
View reviewed changes
parquet/src/file.rs Outdated Show resolved Hide resolved
melotik
melotik reviewed Mar 10, 2023
View reviewed changes
parquet/src/decoder.rs Outdated Show resolved Hide resolved
Robin Bernon and others added 18 commits March 13, 2023 08:48
Parquet sink now working! Tested on EnsNames spkg and it works!
9b59a7b
Hooking up parquet sink to messari cli + big refactor
f4be8fe
Getting rid of extra parquet folder in output filepath
db256c9
ewfiwn
f9c1309
Adding proper enum support. Also improved test setup to start generat…
6cc2c0f 
…ing random data samples for testing the sink
Adding test data derive
0891251
Cleanup
b6c4362
More cleanup
f05e890
@robinbernon
Delete .DS_Store
98da755
@robinbernon
Delete .DS_Store
8c54da7
@robinbernon
Delete .DS_Store
a7f6c10
@robinbernon
Delete .DS_Store
3e8ce50
@robinbernon
Delete .DS_Store
e4a06f6
@robinbernon
Delete .DS_Store
1d493d9
Add support for multiple output files in sink, and also added general…
4aea18e 
… support for oneof type.
Adding oneof support and adding initial macro code for easy end to en…
25ce07e 
…d testing
Adding all the code necessary for setting up the pipeline for substre…
b5e856b 
…ams uploads
Adding cmds to support config file accompanyment for pipeline spkg pr…
817dff5 
…ocessing
@jaimehgb jaimehgb mentioned this pull request May 24, 2023
Merged
@jaimehgb jaimehgb marked this pull request as ready for review July 7, 2023 16:04
@this-username-is-taken this-username-is-taken merged commit 561e9f3 into messari:master Aug 14, 2023
1 of 3 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Reviewers

@melotik melotik melotik left review comments

@jaimehgb jaimehgb jaimehgb left review comments

@this-username-is-taken this-username-is-taken Awaiting requested review from this-username-is-taken

Assignees
No one assigned
Labels
None yet
Projects
None yet
Milestone
No milestone
Development

Successfully merging this pull request may close these issues.
4 participants
@robinbernon @jaimehgb @melotik @this-username-is-taken