feat: increase range of python FFI support #870
Conversation
sxlijin
commented
Aug 7, 2024
•
sxlijin
commented
- aarch64-linux now targets glibc 2.24, which should now support amazon linux 2 (which relies on glibc 2.26; we might be able to make 2.17 work but i have more concerns about that)
- add musl builds, which will allow alpine to work
|
The latest updates on your projects. Learn more about Vercel for Git ↗︎
|
![greptile-apps[bot]](https://avatars.githubusercontent.com/in/867647?s=60&v=4){kind=small}
There was a problem hiding this comment.
PR Summary
The pull request focuses on rolling down the glibc version used for aarch64-linux to ensure compatibility.
- Updated
.github/workflows/build-ruby-release.reusable.yamlto includerb-sys-dock-setupfor aarch64-linux. - Modified
.github/workflows/release.ymlto adjust the build matrix for aarch64-unknown-linux-gnu. - Ensured
cargo install -f [email protected]in.github/workflows/release.ymlfor WASM builds. - Adjusted
pnpm buildcommands in.github/workflows/primary.ymlto align with the new glibc version. - Verified
manylinuxsettings in.github/workflows/build-python-release.reusable.yamlfor aarch64 targets.
No file(s) reviewed, no comment(s)
Edit PR Review Bot Settings
There was a problem hiding this comment.
PR Summary
(updates since last review)
No major changes found since the last review.
No file(s) reviewed, no comment(s)
Edit PR Review Bot Settings
There was a problem hiding this comment.
PR Summary
(updates since last review)
The pull request focuses on rolling down the glibc version used for aarch64-linux in the workflows.
- Updated
containerimage toghcr.io/rust-cross/manylinux_2_28-cross:aarch64in.github/workflows/release.ymlfor aarch64-unknown-linux-gnu builds. - Adjusted
cargo_argsto include-p baml-typescript-ffi -p baml-python-ffifor aarch64-unknown-linux-gnu in.github/workflows/release.yml. - No changes detected in
.github/workflows/primary.yml.
These changes ensure compatibility with the specified glibc version for aarch64-linux builds.
No file(s) reviewed, no comment(s)
Edit PR Review Bot Settings
There was a problem hiding this comment.
PR Summary
(updates since last review)
No major changes found since the last review.
No file(s) reviewed, no comment(s)
Edit PR Review Bot Settings
There was a problem hiding this comment.
PR Summary
(updates since last review)
The pull request focuses on adjusting the glibc version for aarch64-linux builds.
- Updated
.github/workflows/build-python-release.reusable.yamlto specifymanylinux: '2_17'for aarch64-unknown-linux-gnu. - Added
rb-sys-dock-setup: ./x86-64_linux-setup.shfor x86_64-linux in.github/workflows/build-ruby-release.reusable.yaml. - Modified
engine/language_client_rubybuild steps to ensure compatibility with the new glibc version. - Ensured all artifacts are correctly uploaded post-build for both Python and Ruby workflows.
No file(s) reviewed, no comment(s)
Edit PR Review Bot Settings
There was a problem hiding this comment.
PR Summary
(updates since last review)
The pull request focuses on adjusting the glibc version for aarch64-linux builds.
- Updated
.github/workflows/build-python-release.reusable.yamlto specifyCFLAGS_aarch64_unknown_linux_gnu: "-D__ARM_ARCH=8"for aarch64-unknown-linux-gnu. - Added
rb-sys-dock-setup: ./x86-64_linux-setup.shfor x86_64-linux in.github/workflows/build-ruby-release.reusable.yaml. - Modified
engine/language_client_rubybuild steps to ensure compatibility with the new glibc version. - Ensured all artifacts are correctly uploaded post-build for both Python and Ruby workflows.
No file(s) reviewed, no comment(s)
Edit PR Review Bot Settings
There was a problem hiding this comment.
PR Summary
(updates since last review)
The pull request focuses on adjusting the glibc version for aarch64-linux builds.
- Updated
.github/workflows/build-python-release.reusable.yamlto specifyCFLAGS_aarch64_unknown_linux_gnu: "-D__ARM_ARCH=8"for aarch64-unknown-linux-gnu. - Added
rb-sys-dock-setup: ./x86-64_linux-setup.shfor x86_64-linux in.github/workflows/build-ruby-release.reusable.yaml. - Modified
engine/language_client_rubybuild steps to ensure compatibility with the new glibc version. - Ensured all artifacts are correctly uploaded post-build for both Python and Ruby workflows.
No file(s) reviewed, no comment(s)
Edit PR Review Bot Settings
There was a problem hiding this comment.
PR Summary
(updates since last review)
The pull request focuses on adjusting the glibc version for aarch64-linux builds.
- Updated
.github/workflows/build-python-release.reusable.yamlto specifyCFLAGS_aarch64_unknown_linux_gnu: "-D__ARM_ARCH=8"for aarch64-unknown-linux-gnu. - Added
rb-sys-dock-setup: ./x86-64_linux-setup.shfor x86_64-linux in.github/workflows/build-ruby-release.reusable.yaml. - Modified
engine/language_client_rubybuild steps to ensure compatibility with the new glibc version. - Ensured all artifacts are correctly uploaded post-build for both Python and Ruby workflows.
No file(s) reviewed, no comment(s)
Edit PR Review Bot Settings
There was a problem hiding this comment.
PR Summary
(updates since last review)
The pull request focuses on adjusting the glibc version for aarch64-linux builds.
- Updated
.github/workflows/build-python-release.reusable.yamlto specifyCFLAGS_aarch64_unknown_linux_gnu: "-D__ARM_ARCH=8"for aarch64-unknown-linux-gnu. - Added
rb-sys-dock-setup: ./x86-64_linux-setup.shfor x86_64-linux in.github/workflows/build-ruby-release.reusable.yaml. - Modified
engine/language_client_rubybuild steps to ensure compatibility with the new glibc version. - Ensured all artifacts are correctly uploaded post-build for both Python and Ruby workflows.
No file(s) reviewed, no comment(s)
Edit PR Review Bot Settings
There was a problem hiding this comment.
PR Summary
(updates since last review)
No major changes found since the last review.
No file(s) reviewed, no comment(s)
Edit PR Review Bot Settings
There was a problem hiding this comment.
PR Summary
(updates since last review)
The pull request focuses on adjusting the glibc version for aarch64-linux to ensure compatibility and stability.
- Updated
/.github/workflows/build-python-release.reusable.yamlto includeCFLAGS_aarch64_unknown_linux_gnufor aarch64 builds. - Modified
/.github/workflows/build-ruby-release.reusable.yamlto handle specific setup steps for aarch64-linux. - Ensured
engine/language_client_pythonandengine/language_client_rubydirectories are correctly configured for the new glibc version. - Added specific environment variables and setup steps to address build issues related to the glibc version downgrade.
No file(s) reviewed, no comment(s)
Edit PR Review Bot Settings
There was a problem hiding this comment.
PR Summary
(updates since last review)
The pull request adds a comment to the engine/.cargo/config.toml file for better context and maintainability without changing the build configuration.
- Added a comment referencing Rust Cargo issue #8607 in
engine/.cargo/config.tomlfor context. - Configuration settings for
aarch64-unknown-linux-musltarget remain unchanged. - No functional changes to the build configuration, ensuring compatibility and proper compilation.
1 file(s) reviewed, no comment(s)
Edit PR Review Bot Settings
There was a problem hiding this comment.
PR Summary
(updates since last review)
The pull request removes the linker configuration for the aarch64-unknown-linux-musl target in engine/.cargo/config.toml to roll down the glibc version used for aarch64-linux.
- Removed
linkerconfiguration fromengine/.cargo/config.tomlforaarch64-unknown-linux-musltarget. - Existing
rustflagssetting remains unchanged. - Important to verify build and runtime behavior for consistency post-change.
1 file(s) reviewed, no comment(s)
Edit PR Review Bot Settings
There was a problem hiding this comment.
PR Summary
(updates since last review)
The pull request modifies the engine/.cargo/config.toml file to remove the linker configuration for the aarch64-unknown-linux-musl target, while keeping the existing rustflags setting unchanged.
- Removed
linkerconfiguration fromengine/.cargo/config.tomlforaarch64-unknown-linux-musltarget. - Existing
rustflagssetting remains unchanged. - Important to verify build and runtime behavior for consistency post-change.
1 file(s) reviewed, no comment(s)
Edit PR Review Bot Settings
There was a problem hiding this comment.
PR Summary
(updates since last review)
The pull request modifies the engine/.cargo/config.toml file to remove the linker configuration for the aarch64-unknown-linux-musl target, while keeping the existing rustflags setting unchanged.
- Removed
linkerconfiguration fromengine/.cargo/config.tomlforaarch64-unknown-linux-musltarget. - Existing
rustflagssetting remains unchanged. - Important to verify build and runtime behavior for consistency post-change.
No file(s) reviewed, no comment(s)
Edit PR Review Bot Settings
There was a problem hiding this comment.
PR Summary
(updates since last review)
The pull request modifies the engine/.cargo/config.toml file to remove the linker configuration for the aarch64-unknown-linux-musl target, while keeping the existing rustflags setting unchanged.
- Removed
linkerconfiguration fromengine/.cargo/config.tomlforaarch64-unknown-linux-musltarget. - Existing
rustflagssetting remains unchanged. - Important to verify build and runtime behavior for consistency post-change.
No file(s) reviewed, no comment(s)
Edit PR Review Bot Settings
There was a problem hiding this comment.
PR Summary
(updates since last review)
The pull request focuses on rolling down the glibc version for aarch64-linux and introduces several Dockerfiles and an install script for dependencies.
- Updated
.mise.tomlto specify Python 3.10 for consistent environment setup. - Added
integ-tests/python/docker-tests/aarch64-unknown-linux-gnu.Dockerfilefor integration tests on aarch64 architecture. - Added
integ-tests/python/docker-tests/aarch64-unknown-linux-musl.Dockerfilefor musl-based aarch64 environment testing. - Added
integ-tests/python/docker-tests/python-3_10.Dockerfileto set up a Python 3.10 environment for integration testing. - Introduced
tools/installscript for streamlined dependency installation (Poetry, Rustup).
Ensure compatibility and functionality across environments, especially with the new glibc version and Python 3.10.
5 file(s) reviewed, 4 comment(s)
Edit PR Review Bot Settings
| RUN pip3 install ./baml_py-0.53.0-cp38-abi3-manylinux_2_17_aarch64.manylinux2014_aarch64.whl | ||
|
|
||
| ENV RUST_LOG=trace | ||
| RUN baml-cli --help |
There was a problem hiding this comment.
style: Add a newline at the end of the file for better formatting.
| RUN pip install ./baml_py-0.53.0-cp38-abi3-manylinux_2_17_aarch64.manylinux2014_aarch64.whl | ||
|
|
||
| ENV RUST_LOG=trace | ||
| RUN baml-cli --help |
There was a problem hiding this comment.
style: Add a newline at the end of the file for better formatting.
| fi | ||
|
|
||
| if [ "${install_poetry:-0}" -eq 1 ]; then | ||
| curl -sSL https://install.python-poetry.org | python3 - |
There was a problem hiding this comment.
style: Consider adding error handling for the Poetry installation command to catch and report any issues during the installation process.
| fi | ||
|
|
||
| if [ "${install_rustup:-0}" -eq 1 ]; then | ||
| curl --proto '=https' --tlsv1.2 -sSf https://sh.rustup.rs | sh -s -- -y |
There was a problem hiding this comment.
style: Consider adding error handling for the Rustup installation command to catch and report any issues during the installation process.
There was a problem hiding this comment.
PR Summary
(updates since last review)
No major changes found since the last review.
No file(s) reviewed, no comment(s)
Edit PR Review Bot Settings
There was a problem hiding this comment.
PR Summary
(updates since last review)
The pull request focuses on adjusting the glibc version for the aarch64-linux platform in the build and release workflows.
- Updated
/.github/workflows/build-python-release.reusable.yamlto setmanylinuxto2_17foraarch64-unknown-linux-gnu. - Modified
/.github/workflows/build-ruby-release.reusable.yamlto includeaarch64-linuxin the build matrix. - Adjusted
/.github/workflows/release.ymlto ensure compatibility with the new glibc version foraarch64-linux. - Ensured all relevant build steps and dependencies are aligned with the new glibc version for
aarch64-linux.
No file(s) reviewed, no comment(s)
Edit PR Review Bot Settings
There was a problem hiding this comment.
PR Summary
(updates since last review)
No major changes found since the last review.
No file(s) reviewed, no comment(s)
Edit PR Review Bot Settings
There was a problem hiding this comment.
PR Summary
(updates since last review)
This pull request focuses on rolling down the glibc version used for aarch64-linux.
- Updated
.github/workflows/build-python-release.reusable.yamlto adjust glibc version for aarch64-linux. - Added
manylinux: 2_24configuration for aarch64-unknown-linux-gnu target inbuild-python-release.reusable.yaml. - Modified
before-script-linuxto handle package installations for different Linux distributions.
No file(s) reviewed, no comment(s)
Edit PR Review Bot Settings
sxlijin
changed the title
There was a problem hiding this comment.
PR Summary
(updates since last review)
The PR aims to roll down the glibc version used for aarch64-linux and remove the Python version specification in .mise.toml.
- Removed Python version specification in
/.mise.toml, which may lead to inconsistencies across environments. - Added
manylinux: 2_24configuration foraarch64-unknown-linux-gnutarget in.github/workflows/build-python-release.reusable.yaml. - Modified
before-script-linuxin.github/workflows/build-python-release.reusable.yamlto handle package installations for different Linux distributions.
1 file(s) reviewed, no comment(s)
Edit PR Review Bot Settings
