Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[VL] Add reader process to shuffle benchmark #6682

Merged
merged 7 commits into from
Aug 6, 2024

Conversation

marin-ma
Copy link
Contributor

@marin-ma marin-ma commented Aug 1, 2024

  • Add --run-shuffle-read command line flag to add shuffle read process after write. Only takes effect when --run-shuffle is set.
  • Add --debug-mode command line flag to enable debug mode (same as spark.gluten.sql.debug)
  • Refined log printing. Fixed coredump when using --help.

@github-actions github-actions bot added CORE works for Gluten Core VELOX labels Aug 1, 2024
Copy link

github-actions bot commented Aug 1, 2024

Thanks for opening a pull request!

Could you open an issue for this pull request on Github Issues?

https://github.com/apache/incubator-gluten/issues

Then could you also rename commit message and pull request title in the following format?

[GLUTEN-${ISSUES_ID}][COMPONENT]feat/fix: ${detailed message}

See also:

Copy link

github-actions bot commented Aug 1, 2024

Run Gluten Clickhouse CI

1 similar comment
Copy link

github-actions bot commented Aug 2, 2024

Run Gluten Clickhouse CI

@FelixYBW
Copy link
Contributor

FelixYBW commented Aug 2, 2024

Thank you! Rong. Is there new parameter, can you add to document?
Also can you add an example of commandline as well as the output to the threads and document if not yet.

@marin-ma marin-ma force-pushed the shuffle-benchmark-add-reader branch from 9d92dc5 to 7a61fac Compare August 5, 2024 03:50
@github-actions github-actions bot added the DOCS label Aug 5, 2024
Copy link

github-actions bot commented Aug 5, 2024

Run Gluten Clickhouse CI

Copy link

github-actions bot commented Aug 5, 2024

Run Gluten Clickhouse CI

Copy link

github-actions bot commented Aug 6, 2024

Run Gluten Clickhouse CI

Copy link

github-actions bot commented Aug 6, 2024

Run Gluten Clickhouse CI

@FelixYBW
Copy link
Contributor

FelixYBW commented Aug 6, 2024

Thank you, @marin-ma

@marin-ma marin-ma merged commit d746f0f into apache:main Aug 6, 2024
44 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
CORE works for Gluten Core DOCS VELOX
Projects
None yet
Development

Successfully merging this pull request may close these issues.

2 participants