Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

[GLUTEN-6091][CH] Avoid using LD_PRELOAD in child process #6092

Merged
merged 5 commits into from
Jun 14, 2024
Merged

[GLUTEN-6091][CH] Avoid using LD_PRELOAD in child process #6092

merged 5 commits into from
Jun 14, 2024

Conversation

baibaichen
Copy link
Contributor

What changes were proposed in this pull request?

(Fixes: #6091)

See #6091 for issue description. This PR does 2 things:

  1. unset LD_PRELOAD
  2. add missing gwp_asan dependency for gluten_spark_functions to avoid mering [GLUTEN-1632][CH]Daily Update Clickhouse Version (20240614) #6083

How was this patch tested?

Using existed UTs

@github-actions GitHub Actions
Copy link

#6091

@github-actions GitHub Actions
Copy link

Run Gluten Clickhouse CI

@github-actions GitHub Actions
Copy link

Run Gluten Clickhouse CI

1 similar comment
@github-actions GitHub Actions
Copy link

Run Gluten Clickhouse CI

@github-actions GitHub Actions
Copy link

Run Gluten Clickhouse CI

@zzcclp zzcclp merged commit a36d307 into apache:main Jun 14, 2024
7 checks passed
@baibaichen baibaichen deleted the feature/fix-6091 branch June 14, 2024 13:12
@baibaichen baibaichen mentioned this pull request Jun 14, 2024
Closed
@GlutenPerfBot
Copy link
Contributor

===== Performance report for TPCDS SF2000 with Velox backend, for reference only ====

query log/native_master_06_14_2024_time.csv log/native_master_06_13_2024_142cf0fbca_time.csv difference percentage
q1 14.80 15.69 0.885 105.98%
q2 14.02 15.39 1.367 109.75%
q3 4.41 4.29 -0.126 97.15%
q4 66.72 65.30 -1.417 97.88%
q5 7.80 7.66 -0.145 98.14%
q6 5.56 7.58 2.021 136.37%
q7 6.63 6.41 -0.221 96.67%
q8 6.16 5.26 -0.898 85.41%
q9 19.48 17.10 -2.372 87.82%
q10 12.02 9.02 -3.005 75.00%
q11 35.94 35.38 -0.563 98.43%
q12 1.54 1.38 -0.165 89.30%
q13 5.30 5.33 0.030 100.57%
q14a 44.07 41.18 -2.888 93.45%
q14b 39.39 41.74 2.355 105.98%
q15 2.58 2.59 0.007 100.26%
q16 37.78 38.11 0.325 100.86%
q17 5.79 5.50 -0.298 94.85%
q18 6.30 6.04 -0.259 95.89%
q19 3.49 2.09 -1.405 59.75%
q20 3.19 1.28 -1.916 39.97%
q21 2.59 0.94 -1.650 36.33%
q22 8.29 9.32 1.025 112.36%
q23a 80.64 79.23 -1.410 98.25%
q23b 100.12 99.56 -0.561 99.44%
q24a 72.87 68.73 -4.137 94.32%
q24b 82.89 72.86 -10.028 87.90%
q25 4.41 3.98 -0.425 90.36%
q26 2.81 2.60 -0.213 92.42%
q27 4.94 2.94 -2.003 59.46%
q28 20.93 23.86 2.931 114.00%
q29 8.26 7.79 -0.470 94.31%
q30 5.96 4.42 -1.542 74.13%
q31 6.50 7.60 1.097 116.89%
q32 1.12 1.07 -0.050 95.57%
q33 5.29 5.60 0.307 105.81%
q34 7.08 5.77 -1.314 81.46%
q35 7.14 6.77 -0.367 94.85%
q36 3.59 3.30 -0.286 92.02%
q37 4.18 4.28 0.098 102.34%
q38 12.19 11.89 -0.309 97.47%
q39a 3.26 4.08 0.824 125.27%
q39b 2.93 2.91 -0.017 99.43%
q40 6.80 4.50 -2.302 66.16%
q41 0.62 0.63 0.007 101.07%
q42 1.13 0.96 -0.164 85.42%
q43 3.80 3.66 -0.144 96.22%
q44 8.86 7.77 -1.090 87.71%
q45 7.86 4.96 -2.897 63.15%
q46 3.36 2.99 -0.372 88.91%
q47 14.02 14.13 0.110 100.79%
q48 4.57 4.18 -0.388 91.50%
q49 9.83 7.23 -2.598 73.56%
q50 24.39 29.75 5.360 121.98%
q51 10.57 8.24 -2.329 77.97%
q52 1.06 0.97 -0.090 91.55%
q53 1.96 1.71 -0.247 87.37%
q54 3.19 3.14 -0.052 98.36%
q55 1.09 1.08 -0.006 99.42%
q56 4.41 4.50 0.083 101.87%
q57 8.26 8.71 0.452 105.47%
q58 2.71 2.43 -0.283 89.57%
q59 13.72 15.31 1.585 111.55%
q60 4.72 4.71 -0.007 99.85%
q61 5.28 5.36 0.081 101.54%
q62 3.83 3.79 -0.043 98.89%
q63 2.14 2.07 -0.069 96.78%
q64 54.22 56.97 2.753 105.08%
q65 14.14 13.73 -0.411 97.09%
q66 3.81 3.37 -0.444 88.36%
q67 361.70 364.30 2.595 100.72%
q68 3.35 3.71 0.363 110.85%
q69 6.55 6.44 -0.105 98.40%
q70 11.94 8.42 -3.518 70.53%
q71 2.22 2.25 0.031 101.38%
q72 188.49 186.39 -2.097 98.89%
q73 2.45 1.94 -0.512 79.08%
q74 21.59 20.93 -0.661 96.94%
q75 24.68 23.12 -1.560 93.68%
q76 9.81 11.41 1.597 116.28%
q77 2.19 1.75 -0.440 79.92%
q78 44.21 44.37 0.155 100.35%
q79 3.58 3.56 -0.016 99.56%
q80 12.70 13.02 0.314 102.47%
q81 8.19 4.43 -3.760 54.11%
q82 7.67 6.17 -1.500 80.44%
q83 1.53 1.47 -0.060 96.10%
q84 2.87 4.22 1.356 147.27%
q85 9.00 6.70 -2.293 74.51%
q86 3.32 3.45 0.130 103.90%
q87 12.23 12.14 -0.088 99.28%
q88 32.04 20.55 -11.498 64.12%
q89 3.29 2.76 -0.523 84.09%
q90 4.28 2.86 -1.422 66.75%
q91 2.56 3.16 0.599 123.34%
q92 1.44 1.30 -0.136 90.53%
q93 30.64 31.91 1.269 104.14%
q94 22.63 22.55 -0.079 99.65%
q9 87.42 88.12 0.708 100.81%
q5 3.76 2.63 -1.125 70.07%
q96 12.15 12.01 -0.141 98.84%
q97 2.15 2.01 -0.136 93.68%
q98 9.15 8.89 -0.253 97.23%
q99 9.15 8.89 -0.253 97.23%
total 1963.09 1909.59 -53.498 97.27%

@GlutenPerfBot
Copy link
Contributor

===== Performance report for TPCH SF2000 with Velox backend, for reference only ====

query log/native_master_06_14_2024_time.csv log/native_master_06_13_2024_142cf0fbc_time.csv difference percentage
q1 34.77 35.57 0.796 102.29%
q2 24.02 23.78 -0.241 99.00%
q3 41.24 39.61 -1.630 96.05%
q4 31.93 32.76 0.830 102.60%
q5 67.70 67.54 -0.163 99.76%
q6 8.30 9.42 1.111 113.38%
q7 79.22 80.69 1.471 101.86%
q8 85.56 84.77 -0.788 99.08%
q9 121.55 120.70 -0.841 99.31%
q10 48.80 45.25 -3.551 92.72%
q11 20.27 19.79 -0.487 97.60%
q12 28.78 27.92 -0.859 97.01%
q13 38.12 39.43 1.312 103.44%
q14 21.84 18.98 -2.859 86.91%
q15 29.87 33.04 3.175 110.63%
q16 14.36 13.85 -0.515 96.42%
q17 103.16 101.57 -1.592 98.46%
q18 149.27 144.97 -4.298 97.12%
q19 15.37 13.79 -1.575 89.75%
q20 31.14 26.68 -4.463 85.67%
q21 258.75 260.83 2.078 100.80%
q22 12.20 13.91 1.711 114.02%
total 1266.23 1254.85 -11.377 99.10%

@GlutenPerfBot
Copy link
Contributor

===== Performance report for TPCDS SF2000 with Velox backend, for reference only ====

query log/native_master_06_15_2024_time.csv log/native_master_06_14_2024_a36d3077d0_time.csv difference percentage
q1 14.77 14.80 0.036 100.24%
q2 15.82 14.02 -1.796 88.65%
q3 4.33 4.41 0.079 101.82%
q4 63.16 66.72 3.552 105.62%
q5 6.40 7.80 1.406 121.98%
q6 3.78 5.56 1.775 146.92%
q7 6.40 6.63 0.227 103.55%
q8 8.39 6.16 -2.231 73.40%
q9 17.14 19.48 2.334 113.62%
q10 12.32 12.02 -0.301 97.55%
q11 35.90 35.94 0.044 100.12%
q12 2.60 1.54 -1.059 59.28%
q13 5.38 5.30 -0.079 98.53%
q14a 42.90 44.07 1.170 102.73%
q14b 42.70 39.39 -3.311 92.25%
q15 2.80 2.58 -0.220 92.14%
q16 39.40 37.78 -1.619 95.89%
q17 4.71 5.79 1.088 123.11%
q18 6.27 6.30 0.031 100.50%
q19 2.22 3.49 1.267 157.00%
q20 2.86 3.19 0.331 111.58%
q21 2.34 2.59 0.249 110.61%
q22 9.24 8.29 -0.951 89.72%
q23a 80.78 80.64 -0.132 99.84%
q23b 102.84 100.12 -2.716 97.36%
q24a 76.15 72.87 -3.287 95.68%
q24b 80.17 82.89 2.724 103.40%
q25 4.39 4.41 0.013 100.30%
q26 2.84 2.81 -0.029 98.99%
q27 6.59 4.94 -1.647 75.00%
q28 22.18 20.93 -1.249 94.37%
q29 7.89 8.26 0.372 104.72%
q30 4.32 5.96 1.636 137.86%
q31 6.14 6.50 0.360 105.87%
q32 1.27 1.12 -0.145 88.57%
q33 5.18 5.29 0.118 102.29%
q34 5.17 7.08 1.918 137.12%
q35 7.26 7.14 -0.127 98.26%
q36 5.88 3.59 -2.290 61.03%
q37 4.02 4.18 0.167 104.15%
q38 12.00 12.19 0.191 101.59%
q39a 3.20 3.26 0.057 101.79%
q39b 2.75 2.93 0.177 106.45%
q40 4.66 6.80 2.143 145.98%
q41 0.57 0.62 0.051 108.89%
q42 1.08 1.13 0.045 104.21%
q43 3.72 3.80 0.078 102.08%
q44 10.89 8.86 -2.021 81.43%
q45 4.40 7.86 3.465 178.83%
q46 3.23 3.36 0.132 104.09%
q47 15.74 14.02 -1.711 89.13%
q48 4.66 4.57 -0.092 98.03%
q49 9.41 9.83 0.418 104.45%
q50 24.38 24.39 0.002 100.01%
q51 8.69 10.57 1.884 121.68%
q52 0.99 1.06 0.068 106.80%
q53 2.19 1.96 -0.237 89.21%
q54 3.23 3.19 -0.041 98.74%
q55 1.03 1.09 0.054 105.24%
q56 6.88 4.41 -2.471 64.10%
q57 8.74 8.26 -0.475 94.56%
q58 2.59 2.71 0.125 104.84%
q59 13.74 13.72 -0.014 99.90%
q60 4.73 4.72 -0.015 99.68%
q61 5.23 5.28 0.047 100.89%
q62 4.26 3.83 -0.433 89.84%
q63 2.13 2.14 0.010 100.45%
q64 55.12 54.22 -0.897 98.37%
q65 14.06 14.14 0.075 100.53%
q66 3.36 3.81 0.454 113.52%
q67 369.05 361.70 -7.340 98.01%
q68 3.47 3.35 -0.123 96.45%
q69 6.38 6.55 0.166 102.61%
q70 9.26 11.94 2.672 128.85%
q71 5.54 2.22 -3.320 40.09%
q72 190.68 188.49 -2.194 98.85%
q73 2.31 2.45 0.142 106.15%
q74 24.28 21.59 -2.698 88.89%
q75 23.24 24.68 1.436 106.18%
q76 9.87 9.81 -0.061 99.38%
q77 2.21 2.19 -0.021 99.04%
q78 44.76 44.21 -0.552 98.77%
q79 3.65 3.58 -0.069 98.11%
q80 12.08 12.70 0.624 105.17%
q81 5.05 8.19 3.141 162.16%
q82 10.03 7.67 -2.364 76.44%
q83 2.33 1.53 -0.802 65.64%
q84 2.75 2.87 0.117 104.26%
q85 8.72 9.00 0.279 103.20%
q86 3.29 3.32 0.026 100.79%
q87 12.39 12.23 -0.164 98.68%
q88 28.82 32.04 3.222 111.18%
q89 3.33 3.29 -0.040 98.79%
q90 4.40 4.28 -0.121 97.25%
q91 2.58 2.56 -0.021 99.19%
q92 1.51 1.44 -0.070 95.38%
q93 32.16 30.64 -1.520 95.27%
q94 23.34 22.63 -0.703 96.99%
q9 89.41 87.42 -1.990 97.77%
q5 3.83 3.76 -0.076 98.03%
q96 12.24 12.15 -0.094 99.23%
q97 2.07 2.15 0.083 104.02%
q98 8.78 9.15 0.368 104.19%
q99 8.78 9.15 0.368 104.19%
total 1976.38 1963.09 -13.289 99.33%

@GlutenPerfBot
Copy link
Contributor

===== Performance report for TPCH SF2000 with Velox backend, for reference only ====

query log/native_master_06_15_2024_time.csv log/native_master_06_14_2024_a36d3077d_time.csv difference percentage
q1 36.35 34.77 -1.575 95.67%
q2 24.02 24.02 -0.003 99.99%
q3 39.94 41.24 1.300 103.26%
q4 32.07 31.93 -0.132 99.59%
q5 71.31 67.70 -3.607 94.94%
q6 10.01 8.30 -1.704 82.97%
q7 80.02 79.22 -0.800 99.00%
q8 86.23 85.56 -0.673 99.22%
q9 118.57 121.55 2.979 102.51%
q10 44.27 48.80 4.537 110.25%
q11 20.63 20.27 -0.360 98.26%
q12 25.58 28.78 3.200 112.51%
q13 39.40 38.12 -1.277 96.76%
q14 19.17 21.84 2.670 113.93%
q15 33.16 29.87 -3.292 90.07%
q16 14.75 14.36 -0.389 97.36%
q17 100.68 103.16 2.481 102.46%
q18 145.65 149.27 3.619 102.48%
q19 13.92 15.37 1.447 110.40%
q20 33.05 31.14 -1.913 94.21%
q21 256.14 258.75 2.608 101.02%
q22 12.33 12.20 -0.134 98.92%
total 1257.25 1266.23 8.983 100.71%

@baibaichen baibaichen mentioned this pull request Jun 18, 2024
Merged
liuneng1994 pushed a commit that referenced this pull request Jun 18, 2024
@baibaichen
[CI] Disable ENABLE_GWP_ASAN (#6119)
ae491a8 
What changes were proposed in this pull request?
Even with #6092, we still get ci failed randomly

https://opencicd.kyligence.com/job/gluten/job/gluten-ci/10132/
https://opencicd.kyligence.com/job/gluten/job/gluten-ci/10133/
Since we need export LD_PRELOAD in shell which casue comiler segment fault.

img_v3_02bv_3404a655-a3f3-4cfa-9c79-1dce19f82c2g

Let's disable ENABLE_GWP_ASAN first

(Fixes: #6091)

How was this patch tested?
Existed UT
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Reviewers

@zzcclp zzcclp zzcclp approved these changes

Assignees
No one assigned
Labels
None yet
Projects
None yet
Milestone
No milestone
Development

Successfully merging this pull request may close these issues.

Unset LD_PRELOAD to avoid pre-loading libch.so in child process of spark
3 participants
@baibaichen @GlutenPerfBot @zzcclp