Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[VL] Support Spark transform_keys, transform_values function #6095

Merged

Conversation

gaoyangxiaozhu
Copy link
Contributor

What changes were proposed in this pull request?

(Please fill in changes proposed in this fix)

(Fixes: #ISSUE-ID)

How was this patch tested?

(Please explain how this patch was tested. E.g. unit tests, integration tests, manual tests)

(If this patch involves UI changes, please attach a screenshot; otherwise, remove this)

Copy link

Thanks for opening a pull request!

Could you open an issue for this pull request on Github Issues?

https://github.com/apache/incubator-gluten/issues

Then could you also rename commit message and pull request title in the following format?

[GLUTEN-${ISSUES_ID}][COMPONENT]feat/fix: ${detailed message}

See also:

Copy link

Run Gluten Clickhouse CI

Copy link

Run Gluten Clickhouse CI

Copy link

Run Gluten Clickhouse CI

Copy link

Run Gluten Clickhouse CI

@gaoyangxiaozhu
Copy link
Contributor Author

@rui-mo / @PHILO-HE help review

Copy link

Run Gluten Clickhouse CI

@gaoyangxiaozhu gaoyangxiaozhu requested a review from PHILO-HE June 18, 2024 08:24
@PHILO-HE PHILO-HE changed the title [VL] Spark transform_keys, transform_values function [VL] Support Spark transform_keys, transform_values function Jun 18, 2024
Copy link
Contributor

@PHILO-HE PHILO-HE left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Looks good! Thanks!

Copy link
Contributor

@rui-mo rui-mo left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks.

@gaoyangxiaozhu
Copy link
Contributor Author

@PHILO-HE / @rui-mo help merge, thanks

@PHILO-HE PHILO-HE merged commit 800cadd into apache:main Jun 18, 2024
39 checks passed
@GlutenPerfBot
Copy link
Contributor

===== Performance report for TPCH SF2000 with Velox backend, for reference only ====

query log/native_6095_time.csv log/native_master_06_17_2024_5b87efa56_time.csv difference percentage
q1 37.28 34.50 -2.780 92.54%
q2 23.84 23.69 -0.149 99.37%
q3 40.85 39.94 -0.908 97.78%
q4 31.90 32.62 0.726 102.28%
q5 68.74 67.62 -1.121 98.37%
q6 9.69 6.63 -3.054 68.47%
q7 82.19 81.08 -1.115 98.64%
q8 85.74 85.44 -0.298 99.65%
q9 117.66 120.83 3.167 102.69%
q10 45.09 46.34 1.247 102.76%
q11 20.01 19.96 -0.054 99.73%
q12 27.79 25.57 -2.225 92.00%
q13 41.42 38.36 -3.063 92.61%
q14 19.34 19.32 -0.015 99.92%
q15 33.01 32.79 -0.222 99.33%
q16 14.31 13.93 -0.376 97.38%
q17 105.18 103.20 -1.988 98.11%
q18 148.58 147.38 -1.204 99.19%
q19 13.74 14.12 0.378 102.75%
q20 30.11 28.78 -1.336 95.56%
q21 262.57 258.28 -4.281 98.37%
q22 12.72 12.23 -0.492 96.13%
total 1271.79 1252.62 -19.163 98.49%

@GlutenPerfBot
Copy link
Contributor

===== Performance report for TPCH SF2000 with Velox backend, for reference only ====

query log/native_master_06_18_2024_time.csv log/native_master_06_17_2024_5b87efa56_time.csv difference percentage
q1 34.68 34.50 -0.183 99.47%
q2 27.07 23.69 -3.378 87.52%
q3 38.23 39.94 1.711 104.48%
q4 36.14 32.62 -3.519 90.26%
q5 71.13 67.62 -3.513 95.06%
q6 7.39 6.63 -0.754 89.80%
q7 84.79 81.08 -3.714 95.62%
q8 84.78 85.44 0.658 100.78%
q9 117.48 120.83 3.346 102.85%
q10 45.02 46.34 1.324 102.94%
q11 22.82 19.96 -2.864 87.45%
q12 27.15 25.57 -1.584 94.17%
q13 39.59 38.36 -1.229 96.90%
q14 18.08 19.32 1.248 106.90%
q15 30.03 32.79 2.762 109.20%
q16 13.98 13.93 -0.042 99.70%
q17 103.10 103.20 0.098 100.10%
q18 144.12 147.38 3.258 102.26%
q19 13.72 14.12 0.399 102.90%
q20 26.04 28.78 2.739 110.52%
q21 263.69 258.28 -5.406 97.95%
q22 12.17 12.23 0.057 100.47%
total 1261.21 1252.62 -8.584 99.32%

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

4 participants