Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[VL] Apply patch to Velox for compilation #3520

Merged
merged 2 commits into from
Oct 26, 2023
Merged

Conversation

rui-mo
Copy link
Contributor

@rui-mo rui-mo commented Oct 25, 2023

What changes were proposed in this pull request?

Removes build script changes to Velox by applying patch from Gluten to Velox.
oap-project/velox#425

How was this patch tested?

under test

@github-actions
Copy link

Thanks for opening a pull request!

Could you open an issue for this pull request on Github Issues?

https://github.com/oap-project/gluten/issues

Then could you also rename commit message and pull request title in the following format?

[GLUTEN-${ISSUES_ID}][COMPONENT]feat/fix: ${detailed message}

See also:

sudo mkdir -p ${velox_home}/third_party/arrow_patches/
sudo cp ${current_dir}/helpers.patch ${velox_home}/third_party/arrow_patches/
cd ${velox_home}
git apply compilation.patch
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

If git apply fails, will the build script fail or continue the build even though the patch is not applied? Could you confirm it? Thanks!

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

If it fails, I suppose whether to continue or to stop, the build process will fail anyway. How do you think?

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

not actually, as it may build the original code?

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I think we can do a check after executing git apply. And leave some message to help developers easily figure out why build failed.

Reference: https://askubuntu.com/questions/29370/how-to-check-if-a-command-succeeded

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Updated. Thanks.

@rui-mo rui-mo force-pushed the wip_apply branch 3 times, most recently from e02288a to c6b06fd Compare October 25, 2023 08:52
Copy link
Contributor

@PHILO-HE PHILO-HE left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Nit:
Rename compilation.patch to some name like modify_velox_build.patch, which can be more read-friendly to developers.

cd ${velox_home}
git apply compilation.patch
if [ $? -ne 0 ]; then
echo "Failed to apply compilation fixes to Velox: $?."
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Please add exit 1 to directly make the build exit. Suppose it is not acceptable to continue the build without the patch applied.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Updated.

@rui-mo rui-mo force-pushed the wip_apply branch 2 times, most recently from da67950 to 622277a Compare October 26, 2023 00:42
current_dir=$1
velox_home=$2
sudo cp ${current_dir}/modify_velox.patch ${velox_home}/
sudo cp ${current_dir}/modify_arrow.patch ${velox_home}/third_party/
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@rui-mo Do we need to git apply modify_arrow.patch like modify_velox.patch in line 74?

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@rui-mo rui-mo requested a review from PHILO-HE October 26, 2023 03:01
Copy link
Contributor

@PHILO-HE PHILO-HE left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM!

@rui-mo rui-mo merged commit eb46721 into apache:main Oct 26, 2023
5 of 13 checks passed
@GlutenPerfBot
Copy link
Contributor

===== Performance report for TPCH SF2000 with Velox backend, for reference only ====

query log/native_3520_time.csv log/native_master_10_25_2023_05650cc34_time.csv difference percentage
q1 34.98 35.20 0.219 100.63%
q2 24.73 24.96 0.226 100.91%
q3 40.01 40.49 0.480 101.20%
q4 37.75 37.34 -0.406 98.92%
q5 71.91 71.45 -0.461 99.36%
q6 7.17 7.39 0.218 103.04%
q7 86.75 86.09 -0.662 99.24%
q8 86.00 85.50 -0.495 99.42%
q9 119.82 121.65 1.830 101.53%
q10 54.17 53.87 -0.301 99.44%
q11 20.05 20.21 0.166 100.83%
q12 27.83 26.96 -0.866 96.89%
q13 48.90 49.50 0.602 101.23%
q14 19.54 18.52 -1.021 94.78%
q15 32.74 33.91 1.179 103.60%
q16 16.25 16.40 0.153 100.94%
q17 102.10 100.59 -1.514 98.52%
q18 149.33 148.05 -1.280 99.14%
q19 16.79 17.18 0.390 102.32%
q20 30.60 29.74 -0.862 97.18%
q21 223.65 222.15 -1.503 99.33%
q22 13.39 13.28 -0.111 99.17%
total 1264.45 1260.44 -4.017 99.68%

@rui-mo rui-mo deleted the wip_apply branch February 20, 2024 03:19
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

5 participants