Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[GLUTEN-1632][CH]Daily Update Clickhouse Version (20240710) #6383

Merged
merged 3 commits into from
Jul 10, 2024

Conversation

kyligence-git
Copy link
Contributor

Auto commit by gluten daily build, please check the build status and merge it if it's green.

Copy link

github-actions bot commented Jul 9, 2024

#1632

Copy link

github-actions bot commented Jul 9, 2024

Run Gluten Clickhouse CI

@baibaichen
Copy link
Contributor

we need merge due to rebase failed with ClickHouse/ClickHouse#64607

@baibaichen baibaichen force-pushed the rebase_ch/20240710 branch from 5356834 to dd6950e Compare July 10, 2024 11:26
Copy link

Run Gluten Clickhouse CI

@baibaichen
Copy link
Contributor

ClickHouse/ClickHouse#54881 之前,tuple 函数创建的 DataTypeTuplehaveExplicitNamesfalse,之后为 true。当我们把 tuple1 cast 成 tuple2 时,Clickhouse 的逻辑如下:

//see FunctionsConversion.cpp
if (from_type->haveExplicitNames() && to_type->haveExplicitNames()){
    // 根据 name 来查找对应的列
} else {
    // 根据索引来查找对应的列    
}

对于下面的 SQL:

create table test (i1 int , i2 int) stored as parquet;

select named_struct('a', i1, 'b', i2) from test where i1 > 0;

实现中会有如下的 cast

from_type to_type
tuple 函数创建,注意条件 i1 > 0 ,所以 i1notnull 上层指定的返回类型
Tuple(i1 Int64, i2 Nullable(Int64)) Tuple(a Nullable(Int64), b Nullable(Int64))

此时,如果 from_type->haveExplicitNames() 为真,则无法根据列名找到对应的列,返回只为 null 。当前 fix 的方法是将 enable_named_columns_in_function_tuple 设置为 false。

新增 TEST(Clickhouse, PR54881)

Copy link
Contributor

@baibaichen baibaichen left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM

@baibaichen baibaichen merged commit e8e93e7 into apache:main Jul 10, 2024
7 checks passed
@GlutenPerfBot
Copy link
Contributor

===== Performance report for TPCDS SF2000 with Velox backend, for reference only ====

query log/native_master_07_10_2024_time.csv log/native_master_07_07_2024_0cb2db3b7d_time.csv difference percentage
q1 15.29 14.83 -0.455 97.03%
q2 13.95 14.17 0.226 101.62%
q3 4.22 4.22 -0.007 99.83%
q4 63.44 61.61 -1.831 97.11%
q5 7.70 7.53 -0.170 97.79%
q6 3.27 2.49 -0.779 76.16%
q7 6.43 4.14 -2.296 64.30%
q8 3.61 4.67 1.063 129.47%
q9 17.20 25.14 7.944 146.20%
q10 11.04 12.48 1.440 113.05%
q11 35.23 36.05 0.821 102.33%
q12 2.44 1.52 -0.919 62.39%
q13 5.60 8.08 2.476 144.19%
q14a 40.98 41.02 0.046 100.11%
q14b 43.17 40.84 -2.332 94.60%
q15 2.78 3.92 1.142 141.13%
q16 38.06 39.15 1.094 102.87%
q17 4.83 5.04 0.210 104.35%
q18 6.24 6.38 0.136 102.18%
q19 2.24 3.06 0.820 136.53%
q20 1.54 1.57 0.034 102.21%
q21 4.56 2.21 -2.344 48.57%
q22 9.19 8.12 -1.070 88.36%
q23a 83.39 83.11 -0.281 99.66%
q23b 106.18 101.98 -4.203 96.04%
q24a 73.27 80.08 6.815 109.30%
q24b 66.71 67.97 1.255 101.88%
q25 4.42 5.72 1.294 129.25%
q26 4.40 4.17 -0.234 94.69%
q27 3.32 7.40 4.077 222.74%
q28 20.93 24.44 3.511 116.78%
q29 6.97 6.97 -0.003 99.96%
q30 7.19 4.25 -2.939 59.12%
q31 6.56 7.87 1.311 120.00%
q32 1.23 1.16 -0.065 94.70%
q33 4.82 4.82 -0.001 99.98%
q34 4.72 3.89 -0.832 82.37%
q35 8.03 6.92 -1.113 86.15%
q36 3.32 3.47 0.150 104.53%
q37 3.68 3.93 0.250 106.80%
q38 11.62 14.86 3.239 127.86%
q39a 3.62 3.34 -0.276 92.38%
q39b 3.08 2.79 -0.283 90.80%
q40 3.65 3.81 0.155 104.25%
q41 0.62 0.62 -0.004 99.41%
q42 0.97 1.01 0.044 104.60%
q43 8.28 3.82 -4.460 46.15%
q44 8.68 8.51 -0.174 98.00%
q45 3.29 3.42 0.138 104.19%
q46 3.42 3.44 0.014 100.41%
q47 14.48 14.39 -0.088 99.39%
q48 4.48 7.07 2.590 157.76%
q49 9.63 9.98 0.354 103.68%
q50 22.95 19.52 -3.435 85.03%
q51 8.78 8.65 -0.137 98.44%
q52 1.05 1.13 0.078 107.40%
q53 1.95 2.10 0.147 107.56%
q54 3.43 3.37 -0.059 98.29%
q55 1.06 0.93 -0.130 87.83%
q56 4.64 4.48 -0.157 96.61%
q57 8.77 8.73 -0.044 99.50%
q58 2.60 8.33 5.725 320.18%
q59 13.81 14.31 0.497 103.60%
q60 4.72 4.86 0.146 103.09%
q61 5.48 5.38 -0.100 98.17%
q62 4.79 3.87 -0.916 80.86%
q63 2.20 2.13 -0.069 96.84%
q64 50.06 49.37 -0.690 98.62%
q65 15.60 13.56 -2.048 86.87%
q66 3.61 3.61 -0.004 99.89%
q67 352.35 346.91 -5.440 98.46%
q68 3.77 3.63 -0.138 96.33%
q69 6.53 10.86 4.324 166.20%
q70 12.80 7.95 -4.852 62.09%
q71 3.93 2.50 -1.430 63.60%
q72 188.46 187.11 -1.346 99.29%
q73 3.51 2.33 -1.173 66.55%
q74 21.34 21.18 -0.155 99.27%
q75 22.96 23.34 0.376 101.64%
q76 9.18 9.20 0.018 100.19%
q77 2.15 2.34 0.194 109.05%
q78 41.75 41.31 -0.440 98.95%
q79 3.67 4.59 0.926 125.25%
q80 11.12 11.07 -0.050 99.55%
q81 5.10 5.19 0.088 101.73%
q82 9.58 9.25 -0.330 96.56%
q83 1.48 1.67 0.193 113.06%
q84 3.09 2.70 -0.392 87.32%
q85 6.79 7.00 0.209 103.07%
q86 3.36 3.23 -0.124 96.30%
q87 12.14 12.27 0.123 101.02%
q88 25.04 25.87 0.835 103.34%
q89 4.74 3.50 -1.237 73.90%
q90 9.35 9.62 0.271 102.90%
q91 2.70 2.48 -0.212 92.15%
q92 1.45 1.35 -0.101 93.02%
q93 27.59 27.74 0.154 100.56%
q94 21.18 20.96 -0.222 98.95%
q9 85.59 84.02 -1.564 98.17%
q5 3.57 3.29 -0.277 92.22%
q96 12.10 12.06 -0.042 99.65%
q97 2.10 2.22 0.119 105.68%
q98 9.68 9.41 -0.270 97.21%
q99 9.68 9.41 -0.270 97.21%
total 1913.61 1915.94 2.332 100.12%

@GlutenPerfBot
Copy link
Contributor

===== Performance report for TPCH SF2000 with Velox backend, for reference only ====

query log/native_master_07_10_2024_time.csv log/native_master_07_09_2024_ca35e47df_time.csv difference percentage
q1 33.46 36.53 3.073 109.18%
q2 28.57 22.47 -6.091 78.68%
q3 38.44 38.91 0.472 101.23%
q4 35.14 31.14 -4.004 88.61%
q5 71.06 70.60 -0.451 99.36%
q6 6.45 7.61 1.163 118.03%
q7 83.24 86.01 2.766 103.32%
q8 88.10 85.73 -2.364 97.32%
q9 119.97 122.14 2.172 101.81%
q10 43.77 46.37 2.597 105.93%
q11 20.45 20.27 -0.183 99.11%
q12 26.84 23.99 -2.850 89.38%
q13 38.42 38.82 0.402 101.05%
q14 20.94 17.56 -3.377 83.87%
q15 30.00 33.88 3.881 112.94%
q16 15.85 14.18 -1.664 89.50%
q17 105.14 102.32 -2.814 97.32%
q18 145.23 149.22 3.988 102.75%
q19 14.76 13.77 -0.989 93.30%
q20 26.50 26.63 0.124 100.47%
q21 267.51 265.27 -2.237 99.16%
q22 12.07 13.09 1.014 108.40%
total 1271.88 1266.51 -5.373 99.58%

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

3 participants