-
Notifications
You must be signed in to change notification settings - Fork 23
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Build wheels on test-infra machines #160
Conversation
|
||
install-and-test: | ||
runs-on: ubuntu-latest | ||
strategy: | ||
fail-fast: false | ||
matrix: | ||
python-version: ['3.8', '3.12'] | ||
python-version: ['3.9'] |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
We move from 3.8-3.12 to just 3.9.
- We can't use 3.8 anymore because
test-infra
doesn't support it. - on top of that, we can only use 3.9 here (for now) because by default,
test-infra
will only generate job on 3.9 on PRs. Generating job for all python versions can be done by labeling the PR withciflow/binaries/all
(I will eventually document this). What we should be doing here is letting thisinstall-and-test
job rely on the exact same python versions that were generated by thegenerate-matrix
job. I haven't found a way to do that yet, but this should be doable, possibly requirement extensions totest-infra
.
For now though, this is OK. We will still be able to checks all necessary python version when we push a new release.
packaging/post_build_script.sh
Outdated
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
These are the same sanity checks that we used to run before, with the addition of the wheel renaming at the end.
This PR moves our linux wheel building process from the default GA machines to the pytorch infra machine from
test-infra
. Testing is unchanged: we download the built wheels artifacts, install the wheels and run the tests on separate machines.This moves ensures that torchcodec is built on the exact same infra as pytorch.
Additionally, this PR adds a renaming step for the wheels: we rename the wheels from "linux_x86_64.whl" to "manylinux_2_17_x86_64.manylinux2014_x86_64.whl" so that they can be properly uploaded on PyPI. In general that renaming step should be done with tools like
auditwheel
. We don't do that and instead do the renaming ourselves. There are multiple reasons to that:check_glibcxx.py
check.