Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[VL] Fix inaccurate calculation of task slot number used by s.g.s.c.b.v.IOThreads #6071

Merged
merged 5 commits into from
Jun 14, 2024

Conversation

zhztheplayer
Copy link
Member

@zhztheplayer zhztheplayer commented Jun 13, 2024

COLUMNAR_VELOX_CONNECTOR_IO_THREADS.defaultValueString is not accurate enough when it's used in backend initialization because:

  1. SQLConf may not be initialized at that time
  2. N in local[N] mode is not counted

The patch fixes the issue.

@apache apache deleted a comment from github-actions bot Jun 13, 2024
@zhztheplayer zhztheplayer changed the title [VL] Fix wrong calculation of task slot number used by s.g.s.c.b.v.IOThreads [VL] Fix inaccurate calculation of task slot number used by s.g.s.c.b.v.IOThreads Jun 13, 2024
@apache apache deleted a comment from github-actions bot Jun 13, 2024
@apache apache deleted a comment from github-actions bot Jun 13, 2024
Copy link

Run Gluten Clickhouse CI

Copy link

Run Gluten Clickhouse CI

JkSelf
JkSelf previously approved these changes Jun 13, 2024
Copy link

Run Gluten Clickhouse CI

@zhztheplayer zhztheplayer merged commit 1065dd3 into apache:main Jun 14, 2024
40 checks passed
@GlutenPerfBot
Copy link
Contributor

===== Performance report for TPCH SF2000 with Velox backend, for reference only ====

query log/native_6071_time.csv log/native_master_06_13_2024_142cf0fbc_time.csv difference percentage
q1 36.05 35.57 -0.488 98.65%
q2 23.65 23.78 0.133 100.56%
q3 39.23 39.61 0.377 100.96%
q4 32.26 32.76 0.500 101.55%
q5 70.68 67.54 -3.139 95.56%
q6 10.69 9.42 -1.273 88.09%
q7 81.71 80.69 -1.017 98.76%
q8 85.51 84.77 -0.743 99.13%
q9 119.51 120.70 1.190 101.00%
q10 44.04 45.25 1.212 102.75%
q11 20.28 19.79 -0.490 97.58%
q12 24.64 27.92 3.276 113.30%
q13 38.16 39.43 1.275 103.34%
q14 19.89 18.98 -0.910 95.43%
q15 33.36 33.04 -0.313 99.06%
q16 14.11 13.85 -0.264 98.13%
q17 103.72 101.57 -2.149 97.93%
q18 144.71 144.97 0.256 100.18%
q19 15.05 13.79 -1.256 91.65%
q20 26.97 26.68 -0.294 98.91%
q21 256.81 260.83 4.022 101.57%
q22 15.01 13.91 -1.099 92.68%
total 1256.05 1254.85 -1.195 99.90%

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

3 participants