Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[HOTFIX][CH] Using sparkMurmurHash3_32 function instead of the murmurHash3_32 for the shuffle hash algorithm #3613

Merged
merged 1 commit into from
Nov 3, 2023

Conversation

zzcclp
Copy link
Contributor

@zzcclp zzcclp commented Nov 3, 2023

What changes were proposed in this pull request?

The ch murmurHash3_32 function is not compatible of the vanilla spark function, using sparkMurmurHash3_32 function instead of it.

(Fixes: #ISSUE-ID)

How was this patch tested?

(Please explain how this patch was tested. E.g. unit tests, integration tests, manual tests)

(If this patch involves UI changes, please attach a screenshot; otherwise, remove this)

…Hash3_32 for the shuffle hash algorithm

The ch murmurHash3_32 function is not compatible of the vanilla spark function, using sparkMurmurHash3_32 function instead of it.
@zzcclp zzcclp requested a review from baibaichen November 3, 2023 09:35
Copy link

github-actions bot commented Nov 3, 2023

Thanks for opening a pull request!

Could you open an issue for this pull request on Github Issues?

https://github.com/oap-project/gluten/issues

Then could you also rename commit message and pull request title in the following format?

[GLUTEN-${ISSUES_ID}][COMPONENT]feat/fix: ${detailed message}

See also:

Copy link

github-actions bot commented Nov 3, 2023

Run Gluten Clickhouse CI

Copy link
Contributor

@baibaichen baibaichen left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM

@zzcclp
Copy link
Contributor Author

zzcclp commented Nov 3, 2023

sparkMurmurHash3_32 149737
cityHash64 150033

@zzcclp zzcclp merged commit 13a0a63 into apache:main Nov 3, 2023
6 checks passed
@GlutenPerfBot
Copy link
Contributor

===== Performance report for TPCH SF2000 with Velox backend, for reference only ====

query log/native_master_11_03_2023_time.csv log/native_master_11_02_2023_78104be3e_time.csv difference percentage
q1 34.17 34.98 0.810 102.37%
q2 25.09 24.86 -0.235 99.06%
q3 39.87 38.38 -1.486 96.27%
q4 37.07 37.54 0.473 101.27%
q5 70.79 71.16 0.370 100.52%
q6 8.97 7.29 -1.681 81.26%
q7 88.68 88.90 0.226 100.25%
q8 87.82 87.43 -0.398 99.55%
q9 121.60 120.87 -0.727 99.40%
q10 52.15 51.83 -0.320 99.39%
q11 20.26 19.61 -0.641 96.84%
q12 27.25 26.28 -0.966 96.45%
q13 48.45 48.19 -0.257 99.47%
q14 18.58 18.42 -0.166 99.11%
q15 33.71 32.90 -0.808 97.60%
q16 15.93 15.99 0.065 100.41%
q17 100.69 101.55 0.866 100.86%
q18 147.73 147.25 -0.480 99.68%
q19 16.82 17.00 0.174 101.03%
q20 32.51 31.82 -0.684 97.89%
q21 221.69 226.44 4.752 102.14%
q22 13.44 13.39 -0.048 99.64%
total 1263.24 1262.07 -1.162 99.91%

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

3 participants