Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[CH][CELEBORN] CHCelebornColumnarBatchSerializer uses AtomicBoolean to identify whether to call close() to avoid calling close() twice situation #6455

Merged
merged 1 commit into from
Jul 15, 2024

Conversation

SteNicholas
Copy link
Member

What changes were proposed in this pull request?

CHCelebornColumnarBatchSerializer should use AtomicBoolean to identify whether to call close() to avoid calling close() twice situation which causes the replication of metrics like readBatchNumRows, numOutputRows.

How was this patch tested?

CI.

Copy link

Thanks for opening a pull request!

Could you open an issue for this pull request on Github Issues?

https://github.com/apache/incubator-gluten/issues

Then could you also rename commit message and pull request title in the following format?

[GLUTEN-${ISSUES_ID}][COMPONENT]feat/fix: ${detailed message}

See also:

@SteNicholas SteNicholas force-pushed the batch-serializer-close branch 2 times, most recently from 94ebbb5 to 091630c Compare July 15, 2024 06:58
…o identify whether to call close() to avoid calling close() twice situation
@SteNicholas SteNicholas force-pushed the batch-serializer-close branch from 091630c to 2a8aa0a Compare July 15, 2024 08:08
Copy link

Run Gluten Clickhouse CI

@SteNicholas
Copy link
Member Author

Ping @taiyang-li, @zzcclp, @zhztheplayer.

@taiyang-li
Copy link
Contributor

Let's wait for ci

Copy link
Contributor

@taiyang-li taiyang-li left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM

@zzcclp zzcclp merged commit a31df44 into apache:main Jul 15, 2024
5 checks passed
@GlutenPerfBot
Copy link
Contributor

===== Performance report for TPCDS SF2000 with Velox backend, for reference only ====

query log/native_master_07_15_2024_time.csv log/native_master_07_14_2024_b1149915c8_time.csv difference percentage
q1 15.02 14.75 -0.273 98.18%
q2 20.66 13.84 -6.813 67.02%
q3 4.53 4.01 -0.520 88.51%
q4 62.26 63.42 1.161 101.86%
q5 7.42 9.28 1.860 125.08%
q6 3.89 2.38 -1.511 61.17%
q7 5.84 6.16 0.328 105.62%
q8 5.53 5.28 -0.248 95.52%
q9 19.54 18.22 -1.322 93.23%
q10 11.53 10.68 -0.853 92.61%
q11 37.39 37.66 0.270 100.72%
q12 1.56 1.61 0.056 103.62%
q13 6.20 5.91 -0.283 95.43%
q14a 40.95 42.17 1.215 102.97%
q14b 39.80 39.05 -0.756 98.10%
q15 4.40 2.57 -1.832 58.36%
q16 38.06 43.13 5.066 113.31%
q17 4.87 5.03 0.158 103.24%
q18 6.27 6.42 0.150 102.38%
q19 2.53 3.19 0.663 126.22%
q20 1.30 1.50 0.205 115.79%
q21 2.32 1.08 -1.238 46.65%
q22 8.38 7.86 -0.521 93.79%
q23a 81.05 82.64 1.592 101.96%
q23b 103.31 101.11 -2.200 97.87%
q24a 71.20 80.01 8.810 112.37%
q24b 69.42 73.31 3.894 105.61%
q25 4.40 4.39 -0.013 99.71%
q26 2.64 4.41 1.770 167.06%
q27 3.20 4.78 1.573 149.12%
q28 20.90 21.90 1.007 104.82%
q29 8.40 6.55 -1.854 77.93%
q30 6.11 4.15 -1.966 67.83%
q31 9.29 6.09 -3.206 65.50%
q32 1.19 1.13 -0.057 95.23%
q33 9.21 4.76 -4.454 51.64%
q34 5.43 8.81 3.379 162.25%
q35 7.78 7.49 -0.293 96.24%
q36 3.92 3.68 -0.244 93.78%
q37 3.96 5.34 1.376 134.74%
q38 11.99 12.24 0.247 102.06%
q39a 3.18 3.22 0.039 101.23%
q39b 2.85 2.68 -0.174 93.89%
q40 3.70 3.81 0.110 102.98%
q41 0.65 0.63 -0.029 95.59%
q42 0.95 0.96 0.011 101.16%
q43 3.69 3.82 0.125 103.38%
q44 11.93 8.56 -3.371 71.74%
q45 3.52 3.64 0.124 103.51%
q46 3.45 7.16 3.712 207.55%
q47 14.27 14.28 0.016 100.11%
q48 3.82 4.61 0.790 120.65%
q49 9.73 10.12 0.393 104.04%
q50 19.73 20.45 0.719 103.64%
q51 8.01 8.01 0.004 100.05%
q52 1.25 3.85 2.604 308.35%
q53 2.01 2.00 -0.010 99.50%
q54 3.29 3.58 0.298 109.08%
q55 1.01 1.00 -0.011 98.87%
q56 4.38 4.90 0.519 111.84%
q57 8.77 9.10 0.328 103.74%
q58 2.68 2.75 0.063 102.35%
q59 10.88 14.07 3.198 129.40%
q60 4.81 4.82 0.009 100.19%
q61 5.36 5.52 0.163 103.04%
q62 3.70 3.98 0.273 107.37%
q63 1.96 2.16 0.200 110.21%
q64 52.29 51.32 -0.973 98.14%
q65 13.69 13.60 -0.088 99.36%
q66 3.71 3.36 -0.353 90.47%
q67 349.74 354.52 4.779 101.37%
q68 3.62 3.78 0.162 104.47%
q69 8.95 6.05 -2.892 67.67%
q70 8.21 10.90 2.699 132.89%
q71 2.47 2.49 0.019 100.78%
q72 183.50 183.94 0.433 100.24%
q73 2.04 2.28 0.235 111.54%
q74 21.00 21.25 0.249 101.18%
q75 23.59 26.18 2.587 110.97%
q76 9.24 9.48 0.240 102.60%
q77 2.11 2.34 0.231 110.93%
q78 38.62 39.07 0.449 101.16%
q79 3.54 3.62 0.076 102.16%
q80 14.05 11.04 -3.007 78.59%
q81 5.67 5.38 -0.291 94.87%
q82 6.65 6.24 -0.409 93.85%
q83 1.63 1.53 -0.105 93.57%
q84 3.00 2.78 -0.221 92.62%
q85 6.97 6.74 -0.231 96.69%
q86 3.41 3.37 -0.044 98.71%
q87 12.02 14.94 2.918 124.28%
q88 31.75 27.43 -4.326 86.38%
q89 3.33 3.23 -0.100 96.99%
q90 4.39 4.31 -0.088 97.99%
q91 6.07 2.61 -3.456 43.04%
q92 1.39 1.39 0.002 100.13%
q93 28.01 28.67 0.653 102.33%
q94 23.36 21.79 -1.575 93.26%
q9 80.97 81.64 0.664 100.82%
q5 3.91 3.23 -0.674 82.75%
q96 12.06 12.33 0.272 102.25%
q97 2.05 2.16 0.110 105.35%
q98 11.34 12.80 1.458 112.85%
q99 11.34 12.80 1.458 112.85%
total 1901.62 1915.44 13.825 100.73%

@GlutenPerfBot
Copy link
Contributor

===== Performance report for TPCH SF2000 with Velox backend, for reference only ====

query log/native_master_07_15_2024_time.csv log/native_master_07_14_2024_b1149915c_time.csv difference percentage
q1 35.45 36.97 1.522 104.29%
q2 24.68 28.41 3.736 115.14%
q3 39.76 42.85 3.097 107.79%
q4 33.45 37.43 3.977 111.89%
q5 75.36 76.21 0.858 101.14%
q6 7.27 9.07 1.798 124.73%
q7 88.26 90.94 2.678 103.03%
q8 89.66 88.91 -0.744 99.17%
q9 124.99 128.89 3.893 103.11%
q10 47.32 50.32 3.000 106.34%
q11 21.19 21.49 0.294 101.39%
q12 24.22 29.24 5.024 120.74%
q13 41.31 40.98 -0.336 99.19%
q14 21.41 23.52 2.114 109.87%
q15 37.16 37.38 0.221 100.60%
q16 14.24 14.93 0.690 104.85%
q17 107.28 108.10 0.815 100.76%
q18 151.48 152.86 1.373 100.91%
q19 15.15 16.41 1.260 108.32%
q20 31.57 30.65 -0.923 97.08%
q21 271.10 274.86 3.761 101.39%
q22 12.35 14.81 2.463 119.94%
total 1314.66 1355.23 40.569 103.09%

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

4 participants