Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

[GLUTEN-5691][CH] Enable merge on local disk first after insert into mergetree #5692

Merged
merged 10 commits into from
May 30, 2024
Merged

[GLUTEN-5691][CH] Enable merge on local disk first after insert into mergetree #5692

merged 10 commits into from
May 30, 2024

Conversation

loneylee
Copy link
Member

@loneylee loneylee commented May 10, 2024 •
edited
Loading

What changes were proposed in this pull request?

(Please fill in changes proposed in this fix)

(Fixes: #5691)

How was this patch tested?

Test by ut and Manual testing.

The tpch10 data set lineitem table test in minio is as follows

merge on insert = true merge on insert=false
minio storage 1 thread √ √
minio storage 15 threads √ √
default storage 1 thread √ √
default storage 15 threads √ √

@github-actions GitHub Actions
Copy link

#5691

@github-actions GitHub Actions
Copy link

Run Gluten Clickhouse CI

1 similar comment
@github-actions GitHub Actions
Copy link

Run Gluten Clickhouse CI

@github-actions GitHub Actions
Copy link

Run Gluten Clickhouse CI

1 similar comment
@github-actions GitHub Actions
Copy link

Run Gluten Clickhouse CI

@github-actions GitHub Actions
Copy link

Run Gluten Clickhouse CI

@github-actions GitHub Actions
Copy link

Run Gluten Clickhouse CI

@loneylee loneylee marked this pull request as ready for review May 15, 2024 07:04
@github-actions GitHub Actions
Copy link

Run Gluten Clickhouse CI

@github-actions GitHub Actions
Copy link

Run Gluten Clickhouse CI

@github-actions GitHub Actions
Copy link

Run Gluten Clickhouse CI

@github-actions GitHub Actions
Copy link

Run Gluten Clickhouse CI

@github-actions GitHub Actions
Copy link

Run Gluten Clickhouse CI

liuneng1994
liuneng1994 approved these changes May 30, 2024
View reviewed changes
Copy link
Contributor

@liuneng1994 liuneng1994 left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM

@zzcclp zzcclp merged commit 6a28f21 into apache:main May 30, 2024
40 checks passed
@loneylee loneylee deleted the 5691 branch May 30, 2024 09:34
@GlutenPerfBot
Copy link
Contributor

===== Performance report for TPCH SF2000 with Velox backend, for reference only ====

query log/native_5692_time.csv log/native_master_05_29_2024_588faae35_time.csv difference percentage
q1 34.26 33.56 -0.700 97.96%
q2 23.94 23.85 -0.085 99.64%
q3 38.14 37.37 -0.770 97.98%
q4 32.79 32.42 -0.369 98.87%
q5 69.01 69.83 0.824 101.19%
q6 7.57 7.54 -0.026 99.66%
q7 80.82 81.75 0.935 101.16%
q8 85.42 86.57 1.145 101.34%
q9 122.82 118.35 -4.467 96.36%
q10 45.63 44.16 -1.475 96.77%
q11 20.64 22.23 1.587 107.69%
q12 28.04 26.72 -1.317 95.30%
q13 51.99 53.60 1.614 103.10%
q14 21.05 17.59 -3.458 83.57%
q15 30.13 32.89 2.758 109.15%
q16 14.33 13.41 -0.925 93.55%
q17 101.79 103.62 1.834 101.80%
q18 144.41 144.53 0.116 100.08%
q19 16.36 13.59 -2.773 83.05%
q20 28.40 29.66 1.251 104.40%
q21 259.26 260.43 1.170 100.45%
q22 12.23 13.84 1.609 113.15%
total 1269.02 1267.50 -1.523 99.88%

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Reviewers

@liuneng1994 liuneng1994 liuneng1994 approved these changes

Assignees
No one assigned
Labels
None yet
Projects
None yet
Milestone
No milestone
Development

Successfully merging this pull request may close these issues.

[CH] Enable merge on local disk first after insert into mergetree
4 participants
@loneylee @GlutenPerfBot @liuneng1994 @zzcclp