Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[GLUTEN-3602][VL] Sed install_awssdk function to setup-centos8.sh#velox on centos8 #3600

Merged
merged 2 commits into from
Nov 3, 2023

Conversation

dcoliversun
Copy link
Contributor

@dcoliversun dcoliversun commented Nov 2, 2023

What changes were proposed in this pull request?

When exec ./get_velox.sh --enable_s3=on & ./build_velox.sh, we have a error with command not found

-- Installing: /usr/local/lib64/pkgconfig/fmt.pc
+ install_awssdk
scripts/setup-centos8.sh: line 118: install_awssdk: command not found
Error: Process completed with exit code 127.

This is caused by lack of install_awssdk function after rebase.

(Fixes: #3602)

How was this patch tested?

Set enable_s3 as ON in github action.

@dcoliversun dcoliversun marked this pull request as draft November 2, 2023 10:26
Copy link

github-actions bot commented Nov 2, 2023

Thanks for opening a pull request!

Could you open an issue for this pull request on Github Issues?

https://github.com/oap-project/gluten/issues

Then could you also rename commit message and pull request title in the following format?

[GLUTEN-${ISSUES_ID}][COMPONENT]feat/fix: ${detailed message}

See also:

Copy link

github-actions bot commented Nov 2, 2023

Run Gluten Clickhouse CI

@dcoliversun
Copy link
Contributor Author

dcoliversun commented Nov 2, 2023

@ulysses-you Apology if bother you. Could you please start this workflow? I want to verify a suspected bug —— get_velox.sh doesn't sed install_awssdk function into setup-centos8.sh in velox

Copy link

github-actions bot commented Nov 2, 2023

Run Gluten Clickhouse CI

@dcoliversun
Copy link
Contributor Author

@ulysses-you Thanks and I submit an issue #3602. I am working on this ticket.

Copy link

github-actions bot commented Nov 3, 2023

Run Gluten Clickhouse CI

@dcoliversun dcoliversun changed the title [ONLY FOR TEST][VL] test build with s3 on centos8 [GLUTEN-3602][VL] Sed install_awssdk function to setup-centos8.sh#velox on centos8 Nov 3, 2023
@dcoliversun dcoliversun marked this pull request as ready for review November 3, 2023 02:49
Copy link

github-actions bot commented Nov 3, 2023

#3602

Copy link

github-actions bot commented Nov 3, 2023

Run Gluten Clickhouse CI

@dcoliversun
Copy link
Contributor Author

@zhouyuan ready to review :)

Copy link

github-actions bot commented Nov 3, 2023

Run Gluten Clickhouse CI

Copy link

github-actions bot commented Nov 3, 2023

Run Gluten Clickhouse CI

@dcoliversun
Copy link
Contributor Author

@zhouyuan Only ubuntu2204-test-spark34 failed in github action. I believe this pr is ok, please review if you have time

Copy link
Contributor

@zhouyuan zhouyuan left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

👍
We should probably turn on HDFS & S3 connectors by default to make the code covered by more tests.
CC: @PHILO-HE

@zhouyuan zhouyuan merged commit 8ee2bac into apache:main Nov 3, 2023
16 checks passed
@zhouyuan
Copy link
Contributor

zhouyuan commented Nov 3, 2023

@dcoliversun thank you for reporting, and also the good fix!

-yuan

@dcoliversun dcoliversun deleted the centos8 branch November 3, 2023 13:05
@GlutenPerfBot
Copy link
Contributor

===== Performance report for TPCH SF2000 with Velox backend, for reference only ====

query log/native_3600_time.csv log/native_master_11_02_2023_78104be3e_time.csv difference percentage
q1 34.50 34.98 0.478 101.38%
q2 24.64 24.86 0.219 100.89%
q3 38.34 38.38 0.039 100.10%
q4 37.85 37.54 -0.315 99.17%
q5 70.94 71.16 0.222 100.31%
q6 8.75 7.29 -1.457 83.34%
q7 85.17 88.90 3.729 104.38%
q8 87.94 87.43 -0.509 99.42%
q9 121.19 120.87 -0.318 99.74%
q10 50.93 51.83 0.897 101.76%
q11 20.01 19.61 -0.393 98.03%
q12 26.71 26.28 -0.431 98.39%
q13 49.38 48.19 -1.187 97.60%
q14 19.20 18.42 -0.779 95.94%
q15 35.06 32.90 -2.157 93.85%
q16 15.94 15.99 0.046 100.29%
q17 101.84 101.55 -0.282 99.72%
q18 147.86 147.25 -0.618 99.58%
q19 16.89 17.00 0.106 100.63%
q20 31.68 31.82 0.148 100.47%
q21 229.17 226.44 -2.732 98.81%
q22 13.46 13.39 -0.067 99.50%
total 1267.43 1262.07 -5.360 99.58%

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

Build Gluten velox thrid party failed with S3 on Centos8
3 participants