Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

[GLUTEN-6887][VL] Daily Update Velox Version (2024_08_26) #7002

Merged
merged 2 commits into from
Aug 26, 2024
Merged

[GLUTEN-6887][VL] Daily Update Velox Version (2024_08_26) #7002

merged 2 commits into from
Aug 26, 2024

Conversation

GlutenPerfBot
Copy link
Contributor

Upstream Velox's New Commits:

d33cdb251 by Jia Ke, Add RowsStreamingWindowBuild to avoid OOM in Window operator (9025)
3d10ccfc1 by Ke, Make processConfigs function pure virtual (10827)
8d45b3c78 by Deepak Majeti, Set CRC32 checksum algorithm for S3 multipart upload (10801)
52c1daaa1 by snadukudy, Add CI migration blog post (10532)
d46459c32 by Yang Zhang, Add write IO metrics and print physical written bytes in PlanNodeStatstoString (10806)
b5cd1e896 by Krishna Pai, Refactor RowTranslationUtil::toElementRows (10820)
eaec1d349 by Zuyu ZHANG, Fix build errors in Presto due to misconfigured generated proto file paths (10807)
2111fb59a by Jialiang Tan, Add disable pool management option to memory manager (10817)
b81dcaa9b by yingsu00, Allow column names to contain space (10784)
b228e0921 by Deepak Majeti, Set ARROW_DEPENDENCY_SOURCE=AUTO for arrow build (10819)

@github-actions GitHub Actions
Copy link

#6887

@zhouyuan
Copy link
Contributor

@GlutenPerfBot benchmark

@Yohahaha Yohahaha mentioned this pull request Aug 26, 2024
Merged
@GlutenPerfBot
Copy link
Contributor Author

ACK, will benchmark TPCH/DS on this pull request

@GlutenPerfBot
Copy link
Contributor Author

===== Performance report for TPCH SF2000 with Velox backend, for reference only ====

query log/native_7002_time.csv log/native_master_08_25_2024_d4d7241432_time.csv difference percentage
q1 39.97 40.89 0.916 102.29%
q2 30.62 30.28 -0.332 98.91%
q3 54.45 53.78 -0.667 98.77%
q4 42.93 42.76 -0.171 99.60%
q5 103.55 103.48 -0.072 99.93%
q6 13.32 12.15 -1.167 91.24%
q7 115.59 119.11 3.528 103.05%
q8 113.30 115.93 2.635 102.33%
q9 169.26 169.87 0.614 100.36%
q10 65.42 65.36 -0.055 99.92%
q11 27.77 27.09 -0.688 97.52%
q12 31.49 29.91 -1.584 94.97%
q13 50.97 51.30 0.332 100.65%
q14 25.05 27.61 2.562 110.23%
q15 55.47 55.28 -0.194 99.65%
q16 19.22 18.91 -0.310 98.39%
q17 132.10 133.84 1.736 101.31%
q18 198.38 197.60 -0.781 99.61%
q19 31.00 27.69 -3.311 89.32%
q20 45.94 43.29 -2.650 94.23%
q21 388.93 384.30 -4.636 98.81%
q22 17.32 17.74 0.415 102.39%
total 1772.06 1768.18 -3.881 99.78%

@GlutenPerfBot
Copy link
Contributor Author

===== Performance report for TPCDS SF2000 with Velox backend, for reference only ====

query log/native_7002_time.csv log/native_master_08_25_2024_d4d7241432_time.csv difference percentage
q1 13.86 13.90 0.046 100.33%
q2 13.95 14.48 0.531 103.81%
q3 4.90 2.87 -2.030 58.59%
q4 70.16 72.17 2.017 102.87%
q5 9.22 9.02 -0.196 97.88%
q6 3.55 4.12 0.568 115.99%
q7 4.92 6.60 1.673 133.98%
q8 3.57 5.23 1.664 146.60%
q9 26.03 24.75 -1.283 95.07%
q10 9.58 8.47 -1.107 88.44%
q11 38.45 39.05 0.600 101.56%
q12 1.39 1.33 -0.057 95.88%
q13 6.48 6.54 0.054 100.83%
q14a 47.41 45.29 -2.117 95.53%
q14b 41.92 44.33 2.415 105.76%
q15 2.70 2.70 0.001 100.03%
q16 45.74 47.27 1.539 103.36%
q17 5.97 5.33 -0.641 89.27%
q18 6.95 6.87 -0.071 98.98%
q19 2.25 2.31 0.061 102.69%
q20 1.51 1.58 0.070 104.65%
q21 1.18 1.26 0.083 107.09%
q22 7.92 7.77 -0.149 98.12%
q23a 106.95 103.18 -3.775 96.47%
q23b 128.56 130.28 1.719 101.34%
q24a 107.42 116.35 8.930 108.31%
q24b 113.21 110.84 -2.362 97.91%
q25 4.22 4.27 0.047 101.12%
q26 3.23 3.13 -0.102 96.85%
q27 4.26 4.04 -0.219 94.86%
q28 33.83 34.34 0.515 101.52%
q29 11.19 11.79 0.594 105.31%
q30 5.11 5.28 0.169 103.31%
q31 7.50 7.74 0.240 103.20%
q32 1.22 1.23 0.008 100.65%
q33 4.51 4.19 -0.329 92.71%
q34 4.01 4.10 0.085 102.11%
q35 8.25 10.54 2.292 127.78%
q36 4.65 4.37 -0.282 93.94%
q37 4.68 4.68 -0.001 99.97%
q38 15.35 14.28 -1.068 93.04%
q39a 3.15 3.70 0.551 117.48%
q39b 3.03 3.09 0.063 102.09%
q40 3.88 3.90 0.020 100.52%
q41 0.63 0.68 0.048 107.51%
q42 0.96 0.96 -0.004 99.57%
q43 4.64 4.78 0.149 103.22%
q44 10.30 10.08 -0.219 97.87%
q45 3.28 3.31 0.027 100.82%
q46 3.76 3.98 0.222 105.90%
q47 18.69 18.69 -0.003 99.98%
q48 5.18 5.21 0.031 100.60%
q49 8.78 8.84 0.059 100.67%
q50 21.57 21.41 -0.155 99.28%
q51 10.12 10.15 0.024 100.24%
q52 1.07 1.03 -0.044 95.92%
q53 2.40 2.68 0.275 111.44%
q54 3.87 4.26 0.384 109.93%
q55 1.10 1.08 -0.025 97.72%
q56 4.24 4.05 -0.195 95.40%
q57 10.62 10.94 0.324 103.05%
q58 2.47 2.64 0.178 107.23%
q59 11.16 10.82 -0.335 97.00%
q60 4.00 4.14 0.149 103.72%
q61 4.15 4.17 0.023 100.55%
q62 5.01 4.59 -0.420 91.61%
q63 2.40 2.42 0.024 101.02%
q64 59.77 59.74 -0.029 99.95%
q65 19.39 17.20 -2.184 88.73%
q66 4.02 4.11 0.091 102.27%
q67 388.90 410.92 22.016 105.66%
q68 3.47 3.70 0.235 106.78%
q69 7.64 5.58 -2.064 73.00%
q70 11.47 11.88 0.408 103.56%
q71 2.34 2.68 0.342 114.65%
q72 217.53 215.16 -2.369 98.91%
q73 2.32 2.24 -0.080 96.55%
q74 23.30 23.90 0.596 102.56%
q75 26.32 26.47 0.155 100.59%
q76 12.90 11.94 -0.954 92.60%
q77 2.34 2.41 0.072 103.09%
q78 49.76 49.66 -0.095 99.81%
q79 4.00 3.90 -0.097 97.58%
q80 12.70 12.78 0.083 100.66%
q81 5.03 5.08 0.048 100.96%
q82 6.95 7.21 0.265 103.81%
q83 1.68 1.68 -0.001 99.96%
q84 2.83 2.84 0.004 100.16%
q85 7.14 8.07 0.928 113.00%
q86 4.13 4.01 -0.117 97.16%
q87 14.56 14.61 0.051 100.35%
q88 21.68 22.02 0.343 101.58%
q89 3.66 3.78 0.122 103.33%
q90 3.44 3.20 -0.247 92.83%
q91 2.57 2.55 -0.015 99.43%
q92 1.43 1.37 -0.054 96.25%
q93 41.21 39.97 -1.245 96.98%
q94 24.47 24.67 0.199 100.81%
q9 89.91 89.98 0.062 100.07%
q5 3.28 3.18 -0.105 96.79%
q96 17.62 17.86 0.249 101.41%
q97 2.12 1.94 -0.183 91.38%
q98 10.42 10.89 0.470 104.51%
q99 10.42 10.89 0.470 104.51%
total 2200.52 2228.71 28.186 101.28%

@zhouyuan zhouyuan merged commit 5db9de5 into apache:main Aug 26, 2024
41 checks passed
@GlutenPerfBot
Copy link
Contributor Author

===== Performance report for TPCDS SF2000 with Velox backend, for reference only ====

query log/native_master_08_26_2024_time.csv log/native_master_08_25_2024_d4d7241432_time.csv difference percentage
q1 13.96 13.90 -0.055 99.61%
q2 14.04 14.48 0.444 103.16%
q3 4.49 2.87 -1.614 64.03%
q4 71.87 72.17 0.303 100.42%
q5 9.59 9.02 -0.567 94.08%
q6 2.12 4.12 2.001 194.45%
q7 4.91 6.60 1.690 134.44%
q8 5.23 5.23 0.006 100.11%
q9 24.12 24.75 0.632 102.62%
q10 9.91 8.47 -1.445 85.42%
q11 37.86 39.05 1.189 103.14%
q12 1.38 1.33 -0.052 96.22%
q13 6.40 6.54 0.135 102.10%
q14a 45.57 45.29 -0.278 99.39%
q14b 42.03 44.33 2.307 105.49%
q15 2.77 2.70 -0.071 97.44%
q16 46.12 47.27 1.154 102.50%
q17 4.94 5.33 0.393 107.96%
q18 6.87 6.87 0.003 100.04%
q19 2.20 2.31 0.119 105.41%
q20 1.69 1.58 -0.112 93.41%
q21 1.21 1.26 0.052 104.32%
q22 8.08 7.77 -0.310 96.16%
q23a 106.12 103.18 -2.949 97.22%
q23b 129.49 130.28 0.788 100.61%
q24a 106.37 116.35 9.988 109.39%
q24b 106.39 110.84 4.451 104.18%
q25 4.25 4.27 0.019 100.44%
q26 3.12 3.13 0.008 100.25%
q27 3.81 4.04 0.228 105.98%
q28 33.27 34.34 1.074 103.23%
q29 11.32 11.79 0.469 104.15%
q30 5.39 5.28 -0.109 97.98%
q31 7.27 7.74 0.470 106.47%
q32 1.33 1.23 -0.099 92.55%
q33 4.62 4.19 -0.430 90.68%
q34 4.36 4.10 -0.258 94.07%
q35 8.97 10.54 1.572 117.52%
q36 5.49 4.37 -1.123 79.56%
q37 5.67 4.68 -0.989 82.56%
q38 13.29 14.28 0.984 107.40%
q39a 3.15 3.70 0.551 117.49%
q39b 2.81 3.09 0.288 110.26%
q40 3.81 3.90 0.090 102.36%
q41 0.64 0.68 0.041 106.41%
q42 1.09 0.96 -0.130 88.10%
q43 4.66 4.78 0.126 102.71%
q44 9.90 10.08 0.180 101.82%
q45 3.25 3.31 0.061 101.86%
q46 3.75 3.98 0.238 106.36%
q47 18.83 18.69 -0.141 99.25%
q48 5.30 5.21 -0.093 98.24%
q49 8.27 8.84 0.570 106.89%
q50 21.62 21.41 -0.208 99.04%
q51 10.08 10.15 0.065 100.64%
q52 1.11 1.03 -0.082 92.64%
q53 2.39 2.68 0.287 111.99%
q54 3.78 4.26 0.474 112.52%
q55 1.06 1.08 0.018 101.69%
q56 4.28 4.05 -0.237 94.48%
q57 10.63 10.94 0.304 102.86%
q58 2.42 2.64 0.220 109.07%
q59 11.09 10.82 -0.260 97.65%
q60 4.04 4.14 0.106 102.64%
q61 4.11 4.17 0.059 101.43%
q62 4.85 4.59 -0.262 94.61%
q63 2.36 2.42 0.061 102.60%
q64 61.07 59.74 -1.328 97.82%
q65 17.84 17.20 -0.637 96.43%
q66 3.89 4.11 0.225 105.80%
q67 393.73 410.92 17.190 104.37%
q68 3.65 3.70 0.058 101.58%
q69 8.32 5.58 -2.735 67.11%
q70 11.28 11.88 0.597 105.29%
q71 2.50 2.68 0.175 106.98%
q72 215.87 215.16 -0.711 99.67%
q73 2.33 2.24 -0.090 96.14%
q74 24.63 23.90 -0.728 97.04%
q75 27.07 26.47 -0.598 97.79%
q76 12.00 11.94 -0.056 99.54%
q77 2.28 2.41 0.134 105.87%
q78 50.11 49.66 -0.449 99.10%
q79 3.86 3.90 0.045 101.17%
q80 12.67 12.78 0.109 100.86%
q81 4.90 5.08 0.178 103.63%
q82 7.21 7.21 0.010 100.13%
q83 1.61 1.68 0.066 104.09%
q84 2.70 2.84 0.144 105.33%
q85 8.10 8.07 -0.031 99.61%
q86 4.06 4.01 -0.051 98.76%
q87 14.83 14.61 -0.212 98.57%
q88 22.06 22.02 -0.040 99.82%
q89 3.80 3.78 -0.019 99.50%
q90 3.25 3.20 -0.055 98.31%
q91 2.56 2.55 -0.000 99.99%
q92 1.42 1.37 -0.040 97.15%
q93 40.04 39.97 -0.066 99.83%
q94 24.90 24.67 -0.235 99.06%
q9 90.24 89.98 -0.267 99.70%
q5 2.69 3.18 0.483 117.96%
q96 17.69 17.86 0.172 100.97%
q97 1.95 1.94 -0.007 99.62%
q98 10.41 10.89 0.478 104.59%
q99 10.41 10.89 0.478 104.59%
total 2194.66 2228.71 34.049 101.55%

@GlutenPerfBot
Copy link
Contributor Author

===== Performance report for TPCH SF2000 with Velox backend, for reference only ====

query log/native_master_08_26_2024_time.csv log/native_master_08_25_2024_d4d7241432_time.csv difference percentage
q1 41.14 40.89 -0.250 99.39%
q2 29.61 30.28 0.674 102.28%
q3 53.63 53.78 0.153 100.29%
q4 42.24 42.76 0.519 101.23%
q5 107.54 103.48 -4.059 96.23%
q6 10.83 12.15 1.324 112.23%
q7 117.95 119.11 1.160 100.98%
q8 120.99 115.93 -5.058 95.82%
q9 170.67 169.87 -0.798 99.53%
q10 66.86 65.36 -1.501 97.76%
q11 26.86 27.09 0.222 100.83%
q12 29.53 29.91 0.377 101.28%
q13 52.01 51.30 -0.714 98.63%
q14 26.47 27.61 1.137 104.29%
q15 54.81 55.28 0.466 100.85%
q16 18.50 18.91 0.413 102.23%
q17 129.54 133.84 4.303 103.32%
q18 200.27 197.60 -2.667 98.67%
q19 26.33 27.69 1.364 105.18%
q20 42.35 43.29 0.946 102.23%
q21 385.88 384.30 -1.584 99.59%
q22 16.01 17.74 1.726 110.78%
total 1770.02 1768.18 -1.847 99.90%

sharkdtu pushed a commit to sharkdtu/gluten that referenced this pull request Nov 11, 2024
@GlutenPerfBot
[GLUTEN-6887][VL] Daily Update Velox Version (2024_08_26) (apache#7002)
d2adc0b 
Upstream Velox's New Commits:

d33cdb251 by Jia Ke, Add RowsStreamingWindowBuild to avoid OOM in Window operator (9025)
3d10ccfc1 by Ke, Make processConfigs function pure virtual (10827)
8d45b3c78 by Deepak Majeti, Set CRC32 checksum algorithm for S3 multipart upload (10801)
52c1daaa1 by snadukudy, Add CI migration blog post (10532)
d46459c32 by Yang Zhang, Add write IO metrics and print physical written bytes in PlanNodeStatstoString (10806)
b5cd1e896 by Krishna Pai, Refactor RowTranslationUtil::toElementRows (10820)
eaec1d349 by Zuyu ZHANG, Fix build errors in Presto due to misconfigured generated proto file paths (10807)
2111fb59a by Jialiang Tan, Add disable pool management option to memory manager (10817)
b81dcaa9b by yingsu00, Allow column names to contain space (10784)
b228e0921 by Deepak Majeti, Set ARROW_DEPENDENCY_SOURCE=AUTO for arrow build (10819)
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Reviewers

@zhouyuan zhouyuan zhouyuan approved these changes

Assignees
No one assigned
Labels
BUILD VELOX
Projects
None yet
Milestone
No milestone
Development

Successfully merging this pull request may close these issues.
2 participants
@GlutenPerfBot @zhouyuan