Adding Mac to S3 non-GPL FFmpeg compile #210
Merged
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
Adds CI workflows so that we build from S3 on Mac arm64 (the architecture for M1 processors). Enabling Mac arm64 builds required changes in several places:
fetch_and_expose_non_gpl_ffmpeg_libs.cmake
to handle Mac OS on arm64. I had to generalize the logic so it can handle both Mac and Linux. This includes pulling the right tar files from S3 and specifying the right library names. (Linux uses .so; Mac uses .dylib.).github/workflows/macos_wheel.yaml
. The goal for this PR is to get it to the state where it builds from the S3 binaries and runs the Python tests. Subsequent PRs will turn this into the actual wheel building and testing..github/workflows/wheel.yaml
tolinux_wheel.yaml
.test/utils.py
so that when testing on Macs, we allow an absolute tolerance of 3 when comparing tensors. This means that we are okay with not having bit-for-bit-equality between Mac and Linux. Based on what I can find, I think trying to maintain bit-for-bit equality between systems and architectures is unrealistic. We will maintain bit-for-bit equality on Linux x86.VideoDecoderOps.cpp
to throw a pybind11 exception into one specific to PyTorch. The pybind11 exception was supposed to become aStopIteration
exception in Python. That worked on Linux, but it segfaulted on Mac. PyTorch does not have an equivalent ofStopIteration
, but it does have one that becomesIndexError
. I think that's close enough. From what I can tell, PyTorch does not always play nice with pybind11 exceptions; see END_HANDLE_TH_ERRORS_PYBIND prevents pybind11 Exception translation pytorch#34172. I opened Issue C++ code should not throw its own exceptions #212 to track using PyTorch errors everywhere in the C++ code.