Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[GLUTEN-6437][BUILD] Fix vcpkg setup-build-dependens.sh for centos #6438

Merged
merged 2 commits into from
Jul 16, 2024

Conversation

wecharyu
Copy link
Contributor

What changes were proposed in this pull request?

Replace the mirrorlist repository of centos with vault.

(Fixes: #6437)

How was this patch tested?

(Please explain how this patch was tested. E.g. unit tests, integration tests, manual tests)
Test the vcpkd setup scripty on the centos7/8 container:

# run container
podman run --name wechar_gluten --cap-add=SYS_PTRACE --cap-add=AUDIT_WRITE --pids-limit 0 -it --rm  -w /root/workspace --network host --entrypoint bash centos:7

# download gluten source code and run setup script
gluten/dev/vcpkg/setup-build-depends.sh

Copy link

#6437

install_centos_7() {
export PATH=/usr/local/bin:$PATH

yum -y install epel-release centos-release-scl
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I think we should keep it, and also update the mirror for epel repo

    rm /etc/yum.repos.d/CentOS-SCLo-scl.repo -f
    sed -i \
    -e 's/^mirrorlist/#mirrorlist/' \
    -e 's/^#baseurl/baseurl/' \
    -e 's/mirror\.centos\.org/vault.centos.org/' \
    /etc/yum.repos.d/CentOS-SCLo-scl-rh.repo

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

There seems no CentOS-SCLo-scl.repo and CentOS-SCLo-scl-rh.repo files in the default centos image.
And vault repository does not have epel-release centos-release-scl packages.

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I mean if we keep this line to install gcc from SCL repo
yum -y install epel-release centos-release-scl

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

It does support SCL
apache/arrow#43127

@wecharyu
Copy link
Contributor Author

@zhouyuan pls take a look again, I have test on centos 7/8, it works fine now.

Copy link
Contributor

@zhouyuan zhouyuan left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

👍

@zhouyuan zhouyuan merged commit 6b4e5a8 into apache:main Jul 16, 2024
39 checks passed
@GlutenPerfBot
Copy link
Contributor

===== Performance report for TPCDS SF2000 with Velox backend, for reference only ====

query log/native_master_07_16_2024_time.csv log/native_master_07_15_2024_a31df44bd6_time.csv difference percentage
q1 15.20 15.02 -0.178 98.83%
q2 13.60 20.66 7.060 151.93%
q3 4.48 4.53 0.049 101.10%
q4 63.93 62.26 -1.667 97.39%
q5 7.57 7.42 -0.153 97.98%
q6 3.62 3.89 0.269 107.44%
q7 5.95 5.84 -0.116 98.04%
q8 5.93 5.53 -0.407 93.15%
q9 23.62 19.54 -4.072 82.76%
q10 11.61 11.53 -0.077 99.34%
q11 36.99 37.39 0.406 101.10%
q12 3.10 1.56 -1.546 50.18%
q13 6.53 6.20 -0.333 94.89%
q14a 42.43 40.95 -1.481 96.51%
q14b 39.15 39.80 0.657 101.68%
q15 2.55 4.40 1.850 172.58%
q16 38.44 38.06 -0.378 99.02%
q17 5.13 4.87 -0.259 94.94%
q18 6.63 6.27 -0.362 94.55%
q19 2.36 2.53 0.167 107.05%
q20 1.39 1.30 -0.089 93.58%
q21 0.97 2.32 1.352 239.56%
q22 7.98 8.38 0.402 105.03%
q23a 82.30 81.05 -1.246 98.49%
q23b 101.76 103.31 1.554 101.53%
q24a 74.42 71.20 -3.223 95.67%
q24b 69.86 69.42 -0.445 99.36%
q25 4.82 4.40 -0.419 91.30%
q26 4.68 2.64 -2.037 56.44%
q27 3.39 3.20 -0.191 94.37%
q28 24.79 20.90 -3.888 84.31%
q29 7.08 8.40 1.323 118.69%
q30 4.44 6.11 1.670 137.61%
q31 7.74 9.29 1.555 120.09%
q32 1.13 1.19 0.059 105.21%
q33 4.96 9.21 4.250 185.67%
q34 3.99 5.43 1.437 135.99%
q35 7.00 7.78 0.781 111.16%
q36 3.62 3.92 0.305 108.42%
q37 3.89 3.96 0.071 101.81%
q38 11.72 11.99 0.271 102.31%
q39a 3.26 3.18 -0.077 97.64%
q39b 4.91 2.85 -2.061 58.05%
q40 4.04 3.70 -0.346 91.44%
q41 0.62 0.65 0.030 104.85%
q42 0.97 0.95 -0.017 98.26%
q43 3.63 3.69 0.059 101.62%
q44 8.67 11.93 3.255 137.53%
q45 4.92 3.52 -1.397 71.59%
q46 3.63 3.45 -0.180 95.05%
q47 14.38 14.27 -0.109 99.24%
q48 3.94 3.82 -0.115 97.09%
q49 9.63 9.73 0.099 101.02%
q50 23.54 19.73 -3.807 83.83%
q51 9.27 8.01 -1.261 86.39%
q52 1.04 1.25 0.213 120.52%
q53 2.00 2.01 0.016 100.82%
q54 3.33 3.29 -0.043 98.70%
q55 0.95 1.01 0.065 106.90%
q56 4.53 4.38 -0.151 96.68%
q57 8.37 8.77 0.399 104.77%
q58 2.77 2.68 -0.092 96.70%
q59 10.83 10.88 0.047 100.44%
q60 7.61 4.81 -2.800 63.22%
q61 5.38 5.36 -0.027 99.50%
q62 4.15 3.70 -0.444 89.30%
q63 2.05 1.96 -0.091 95.57%
q64 48.96 52.29 3.330 106.80%
q65 14.34 13.69 -0.654 95.44%
q66 3.39 3.71 0.322 109.52%
q67 352.42 349.74 -2.684 99.24%
q68 3.75 3.62 -0.136 96.39%
q69 6.34 8.95 2.602 141.01%
q70 13.66 8.21 -5.449 60.09%
q71 2.45 2.47 0.014 100.57%
q72 185.22 183.50 -1.719 99.07%
q73 2.39 2.04 -0.352 85.29%
q74 24.05 21.00 -3.058 87.29%
q75 23.40 23.59 0.185 100.79%
q76 9.46 9.24 -0.219 97.68%
q77 2.10 2.11 0.007 100.33%
q78 43.63 38.62 -5.012 88.51%
q79 3.59 3.54 -0.043 98.81%
q80 11.36 14.05 2.686 123.64%
q81 5.24 5.67 0.426 108.12%
q82 6.22 6.65 0.428 106.88%
q83 3.57 1.63 -1.942 45.67%
q84 2.69 3.00 0.315 111.75%
q85 7.02 6.97 -0.054 99.23%
q86 3.35 3.41 0.061 101.82%
q87 14.01 12.02 -1.993 85.77%
q88 23.81 31.75 7.946 133.38%
q89 3.27 3.33 0.065 101.99%
q90 4.67 4.39 -0.274 94.13%
q91 7.95 6.07 -1.883 76.31%
q92 1.41 1.39 -0.019 98.67%
q93 28.12 28.01 -0.104 99.63%
q94 20.91 23.36 2.450 111.72%
q9 82.08 80.97 -1.113 98.64%
q5 3.52 3.91 0.390 111.09%
q96 12.21 12.06 -0.158 98.71%
q97 2.14 2.05 -0.087 95.93%
q98 8.62 11.34 2.721 131.56%
q99 8.62 11.34 2.721 131.56%
total 1910.53 1901.62 -8.915 99.53%

@GlutenPerfBot
Copy link
Contributor

===== Performance report for TPCH SF2000 with Velox backend, for reference only ====

query log/native_master_07_16_2024_time.csv log/native_master_07_15_2024_a31df44bd_time.csv difference percentage
q1 39.18 35.45 -3.727 90.49%
q2 24.54 24.68 0.135 100.55%
q3 42.04 39.76 -2.284 94.57%
q4 32.49 33.45 0.961 102.96%
q5 73.76 75.36 1.593 102.16%
q6 8.61 7.27 -1.335 84.49%
q7 84.45 88.26 3.816 104.52%
q8 88.55 89.66 1.110 101.25%
q9 127.33 124.99 -2.340 98.16%
q10 50.38 47.32 -3.062 93.92%
q11 20.55 21.19 0.646 103.14%
q12 26.02 24.22 -1.802 93.07%
q13 40.23 41.31 1.085 102.70%
q14 19.02 21.41 2.383 112.53%
q15 34.96 37.16 2.196 106.28%
q16 14.50 14.24 -0.255 98.24%
q17 106.41 107.28 0.879 100.83%
q18 150.12 151.48 1.360 100.91%
q19 15.27 15.15 -0.126 99.18%
q20 29.29 31.57 2.282 107.79%
q21 267.34 271.10 3.755 101.40%
q22 12.79 12.35 -0.439 96.57%
total 1307.83 1314.66 6.831 100.52%

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

vcpkg setup-build-depends.sh not work for centos 7/8
3 participants