Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[GLUTEN-3779][CH] Fix core dump when executing sql with runtime filter #3781

Merged
merged 1 commit into from
Nov 20, 2023

Conversation

zzcclp
Copy link
Contributor

@zzcclp zzcclp commented Nov 20, 2023

What changes were proposed in this pull request?

When there are empty input blocks in the final stage with the bloom_filter_agg, it will be core dump for the CH backend.
image
image

The RC is: when merging values in the final stage, the input data maybe a non-init AggregateFunctionGroupBloomFilterData, it will use the wrong filter size and filter_hashes values to init the first AggregateFunctionGroupBloomFilterData, which leads to set the wrong filter size when merging values.

Close #3779.

(Fixes: #3779)

How was this patch tested?

(Please explain how this patch was tested. E.g. unit tests, integration tests, manual tests)

(If this patch involves UI changes, please attach a screenshot; otherwise, remove this)

@zzcclp zzcclp requested a review from binmahone November 20, 2023 08:50
Copy link

#3779

Copy link

Run Gluten Clickhouse CI

When there are more than one input row in the final stage with the bloom_filter_agg, it will be core dump for the CH backend.

The RC is: when merging values in the final stage, the input data maybe a non-init AggregateFunctionGroupBloomFilterData, it will use the wrong filter size and filter_hashes values to init the first AggregateFunctionGroupBloomFilterData, which leads to set the wrong filter size when merging values.

Close apache#3779.
Copy link

Run Gluten Clickhouse CI

@binmahone binmahone merged commit 60fc2a0 into apache:main Nov 20, 2023
6 checks passed
@GlutenPerfBot
Copy link
Contributor

===== Performance report for TPCH SF2000 with Velox backend, for reference only ====

query log/native_master_11_20_2023_time.csv log/native_master_11_19_2023_5f5d18abe_time.csv difference percentage
q1 34.37 34.50 0.132 100.38%
q2 24.70 24.57 -0.135 99.45%
q3 35.40 37.14 1.738 104.91%
q4 36.40 36.74 0.343 100.94%
q5 68.86 70.93 2.070 103.01%
q6 7.01 7.09 0.075 101.07%
q7 85.39 83.62 -1.775 97.92%
q8 87.44 85.87 -1.566 98.21%
q9 124.93 126.22 1.288 101.03%
q10 46.83 46.75 -0.076 99.84%
q11 19.99 19.85 -0.146 99.27%
q12 24.57 25.57 0.999 104.06%
q13 44.85 45.26 0.411 100.92%
q14 18.77 18.39 -0.381 97.97%
q15 27.13 28.36 1.229 104.53%
q16 15.41 15.85 0.435 102.82%
q17 102.21 100.27 -1.936 98.11%
q18 149.83 147.67 -2.161 98.56%
q19 13.48 13.00 -0.480 96.44%
q20 28.10 26.75 -1.355 95.18%
q21 220.09 222.79 2.699 101.23%
q22 12.89 13.20 0.309 102.39%
total 1228.65 1230.37 1.717 100.14%

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

[CH] Fix core dump when executing sql with runtime filter
3 participants