Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Update wheels.yaml for recent CI arch changes + Python 3.12 #329

Merged
merged 33 commits into from
Jun 20, 2024
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
33 commits
Select commit Hold shift + click to select a range
1eb7808
Add temporary section to `wheels.yaml`.
dhermes Jun 19, 2024
659c1ef
Temporarily skip jobs other than `sdist`.
dhermes Jun 19, 2024
8de427f
Switch to `python -m build` for `sdist`.
dhermes Jun 19, 2024
1e15d62
Add missing (`build`) dependency for `sdist`.
dhermes Jun 19, 2024
317fed7
Use latest and pinned actions in `wheels.yaml`.
dhermes Jun 19, 2024
817d433
Debug Windows now.
dhermes Jun 19, 2024
cc4e177
Attempt just intel on macOS (upgrade to macOS 14 images, which are ARM).
dhermes Jun 19, 2024
2fa7b3c
Temporarily set `CMAKE_OSX_ARCHITECTURES` explicitly.
dhermes Jun 19, 2024
9a85c0a
Add "inspect" command.
dhermes Jun 19, 2024
d4555c7
Start with `arm64`.
dhermes Jun 19, 2024
e193d46
Change `CMAKE_OSX_DEPLOYMENT_TARGET`.
dhermes Jun 19, 2024
e7664a6
Set later deployment target.
dhermes Jun 19, 2024
b9586be
Show `gfortran` information.
dhermes Jun 19, 2024
c0e1259
All the way to 14.0 (for now).
dhermes Jun 19, 2024
a860f23
Switch gears to `linux`.
dhermes Jun 19, 2024
121afdf
Use different runner for intel macOS.
dhermes Jun 19, 2024
7797ea6
Try again `macos-13` is intel.
dhermes Jun 19, 2024
9e26ec9
Hardcode SDK to 13.0
dhermes Jun 19, 2024
f428f29
Back to arm64 on macOS, no modifications.
dhermes Jun 20, 2024
4f780e5
Set `MACOSX_DEPLOYMENT_TARGET` in `CIBW_ENVIRONMENT`.
dhermes Jun 20, 2024
2bbf246
Try x86_64 build in macos-14.
dhermes Jun 20, 2024
a11429d
Use an explicit PATH for x86_64 homebrew.
dhermes Jun 20, 2024
b53debf
Handle `CMAKE_OSX_DEPLOYMENT_TARGET` same as `CMAKE_Fortran_COMPILER`.
dhermes Jun 20, 2024
cba8064
Use separate workflow jobs for x86 / arm on macOS.
dhermes Jun 20, 2024
0e22d13
Remove all `false` job skips.
dhermes Jun 20, 2024
1a18c74
Set `CMAKE_OSX_DEPLOYMENT_TARGET` env in doctests CI step.
dhermes Jun 20, 2024
c4ad5f8
Remove temporary block for this branch.
dhermes Jun 20, 2024
8c2cd2f
Remove script that is no longer needed.
dhermes Jun 20, 2024
000f8d5
Use `macos-14` in `macos.yaml` job.
dhermes Jun 20, 2024
2b1c0cd
Include `scipy` and `sympy` in `CIBW_TEST_REQUIRES`.
dhermes Jun 20, 2024
f843f77
Go back to `macos-13`.
dhermes Jun 20, 2024
f344322
Use job-level `env` for macOS CI job.
dhermes Jun 20, 2024
69c425b
autoformat `wheels.yaml`.
dhermes Jun 20, 2024
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
2 changes: 2 additions & 0 deletions .github/workflows/macos.yaml
Original file line number Diff line number Diff line change
Expand Up @@ -31,6 +31,8 @@ on:
jobs:
ci:
runs-on: macos-13
env:
CMAKE_OSX_DEPLOYMENT_TARGET: "13.0"
Copy link
Owner Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

See https://github.com/dhermes/bezier/pull/318/files#r1643786071 for earlier context on this (this is meant to be temporary!)

strategy:
matrix:
python-version:
Expand Down
75 changes: 64 additions & 11 deletions .github/workflows/wheels.yaml
Original file line number Diff line number Diff line change
Expand Up @@ -68,18 +68,16 @@ jobs:
LD_LIBRARY_PATH=/project/.nox/.cache/libbezier-release/usr/lib
TARGET_NATIVE_ARCH=OFF
CIBW_TEST_COMMAND: pytest {project}/tests/unit
CIBW_TEST_REQUIRES: pytest
CIBW_TEST_REQUIRES: "pytest scipy sympy"
CIBW_SKIP: "*musllinux*"

- uses: actions/upload-artifact@65462800fd760344b1a7b4382951275a0abb4808 # v4.3.3
with:
name: ${{ matrix.artifact-name }}
path: ./wheelhouse/*.whl

macos-intel:
# NOTE: For now `scripts/macos/build-wheels-arm.sh` must be used for
# the ARM (M1) wheels.
runs-on: macos-13
macos-arm64:
runs-on: macos-14
Copy link
Owner Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I'd consider making a build matrix for this to consolidate


steps:
- uses: actions/checkout@692973e3d937129bcbf40652eb9f2f61becf3332 # v4.1.7
Expand All @@ -102,21 +100,72 @@ jobs:
nox --session libbezier-release

- name: Build wheels
uses: pypa/cibuildwheel@932529cab190fafca8c735a551657247fa8f8eaf #v2.19.1
uses: pypa/cibuildwheel@932529cab190fafca8c735a551657247fa8f8eaf # v2.19.1
env:
BEZIER_INSTALL_PREFIX: ./.nox/.cache/libbezier-release/usr
CIBW_ARCHS: arm64
CIBW_BEFORE_BUILD: |
python -m pip install numpy
CIBW_BUILD: >-
cp310-*
cp311-*
cp312-*
CIBW_ENVIRONMENT: >-
MACOSX_DEPLOYMENT_TARGET=14.0
Comment on lines +113 to +114
Copy link
Owner Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This is really unfortunate but Homebrew installs binaries that target the current OS (which is 14.0)

This results in delocate failures when trying to target the minimum SDK version supported by Python. (M1 / arm64 first started in macOS 11.0 Big Sur)

...
  + delocate-wheel --require-archs arm64 -w /private/var/folders/dm/88b38gj92jj53dgxdsm12qf00000gn/T/cibw-run-lxw12ek6/cp310-macosx_arm64/repaired_wheel -v /private/var/folders/dm/88b38gj92jj53dgxdsm12qf00000gn/T/cibw-run-lxw12ek6/cp310-macosx_arm64/built_wheel/bezier-2023.7.29.dev1-cp310-cp310-macosx_11_0_arm64.whl
  INFO:delocate.delocating:Copying library /Users/runner/work/bezier/bezier/.nox/.cache/libbezier-release/usr/lib/libbezier.2023.7.28.dylib to bezier/.dylibs/libbezier.2023.7.28.dylib
  INFO:delocate.delocating:Copying library /opt/homebrew/Cellar/gcc/14.1.0_1/lib/gcc/current/libgfortran.5.dylib to bezier/.dylibs/libgfortran.5.dylib
  INFO:delocate.delocating:Copying library /opt/homebrew/Cellar/gcc/14.1.0_1/lib/gcc/current/libquadmath.0.dylib to bezier/.dylibs/libquadmath.0.dylib
  INFO:delocate.delocating:Copying library /opt/homebrew/Cellar/gcc/14.1.0_1/lib/gcc/current/libgcc_s.1.1.dylib to bezier/.dylibs/libgcc_s.1.1.dylib
  INFO:delocate.delocating:Modifying install name in bezier/.dylibs/libbezier.2023.7.28.dylib from /opt/homebrew/opt/gcc/lib/gcc/current/libgfortran.5.dylib to @loader_path/libgfortran.5.dylib
  INFO:delocate.delocating:Modifying install name in bezier/.dylibs/libgfortran.5.dylib from @rpath/libgcc_s.1.1.dylib to @loader_path/libgcc_s.1.1.dylib
  INFO:delocate.delocating:Modifying install name in bezier/_speedup.cpython-310-darwin.so from @rpath/libbezier.2023.dylib to @loader_path/.dylibs/libbezier.2023.7.28.dylib
  INFO:delocate.delocating:Modifying install name in bezier/.dylibs/libbezier.2023.7.28.dylib from /opt/homebrew/opt/gcc/lib/gcc/current/libquadmath.0.dylib to @loader_path/libquadmath.0.dylib
  INFO:delocate.delocating:Modifying install name in bezier/.dylibs/libgfortran.5.dylib from @rpath/libquadmath.0.dylib to @loader_path/libquadmath.0.dylib
  Fixing: /private/var/folders/dm/88b38gj92jj53dgxdsm12qf00000gn/T/cibw-run-lxw12ek6/cp310-macosx_arm64/built_wheel/bezier-2023.7.29.dev1-cp310-cp310-macosx_11_0_arm64.whl
  Traceback (most recent call last):
    File "/private/var/folders/dm/88b38gj92jj53dgxdsm12qf00000gn/T/cibw-run-lxw12ek6/cp310-macosx_arm64/build/venv/bin/delocate-wheel", line 8, in <module>
      sys.exit(main())
    File "/private/var/folders/dm/88b38gj92jj53dgxdsm12qf00000gn/T/cibw-run-lxw12ek6/cp310-macosx_arm64/build/venv/lib/python3.10/site-packages/delocate/cmd/delocate_wheel.py", line 110, in main
      copied = delocate_wheel(
    File "/private/var/folders/dm/88b38gj92jj53dgxdsm12qf00000gn/T/cibw-run-lxw12ek6/cp310-macosx_arm64/build/venv/lib/python3.10/site-packages/delocate/delocating.py", line 1004, in delocate_wheel
      out_wheel_fixed = _check_and_update_wheel_name(
    File "/private/var/folders/dm/88b38gj92jj53dgxdsm12qf00000gn/T/cibw-run-lxw12ek6/cp310-macosx_arm64/build/venv/lib/python3.10/site-packages/delocate/delocating.py", line 839, in _check_and_update_wheel_name
      raise DelocationError(
  delocate.libsana.DelocationError: Library dependencies do not satisfy target MacOS version 11.0:
  /private/var/folders/dm/88b38gj92jj53dgxdsm12qf00000gn/T/tmp0b6da8na/wheel/bezier/.dylibs/libbezier.2023.7.28.dylib has a minimum target of 13.0
  /private/var/folders/dm/88b38gj92jj53dgxdsm12qf00000gn/T/tmp0b6da8na/wheel/bezier/.dylibs/libgfortran.5.dylib has a minimum target of 14.0
  /private/var/folders/dm/88b38gj92jj53dgxdsm12qf00000gn/T/tmp0b6da8na/wheel/bezier/.dylibs/libquadmath.0.dylib has a minimum target of 14.0

CIBW_TEST_COMMAND: pytest {project}/tests/unit
CIBW_TEST_REQUIRES: pytest
CIBW_TEST_REQUIRES: "pytest scipy sympy"

- uses: actions/upload-artifact@65462800fd760344b1a7b4382951275a0abb4808 # v4.3.3
with:
name: macos-intel
name: macos-arm64
path: ./wheelhouse/*.whl

macos-x86-64:
runs-on: macos-14

steps:
- uses: actions/checkout@692973e3d937129bcbf40652eb9f2f61becf3332 # v4.1.7

- uses: actions/setup-python@82c7e631bb3cdc910f68e0081d67478d79c6982d # v5.1.0
with:
python-version: "3.12"

- name: Install `homebrew` in x86_64 arch
run: |
arch -x86_64 /bin/bash -c "$(curl -fsSL https://raw.githubusercontent.com/Homebrew/install/HEAD/install.sh)"

- name: Install `gcc` in x86_64 `homebrew
run: |
arch -x86_64 /usr/local/bin/brew install gcc
Comment on lines +133 to +139
Copy link
Owner Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.


- name: Build `libbezier`
env:
TARGET_NATIVE_ARCH: "OFF"
# NOTE: Could also use $(brew --prefix gcc)/bin/gfortran
CMAKE_FORTRAN_COMPILER: /usr/local/bin/gfortran
Comment on lines +144 to +145
Copy link
Owner Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Emphasis!

run: |
python3.12 -m pip install cmake nox
nox --session libbezier-release

- name: Build wheels
uses: pypa/cibuildwheel@932529cab190fafca8c735a551657247fa8f8eaf # v2.19.1
env:
BEZIER_INSTALL_PREFIX: ./.nox/.cache/libbezier-release/usr
CIBW_ARCHS: x86_64
CIBW_BEFORE_BUILD: |
python -m pip install numpy
CIBW_BUILD: >-
cp310-*
cp311-*
cp312-*
CIBW_ENVIRONMENT: >-
MACOSX_DEPLOYMENT_TARGET=14.0
CIBW_TEST_COMMAND: pytest {project}/tests/unit
CIBW_TEST_REQUIRES: "pytest scipy sympy"

- uses: actions/upload-artifact@65462800fd760344b1a7b4382951275a0abb4808 # v4.3.3
with:
name: macos-x86-64
path: ./wheelhouse/*.whl

sdist:
Expand All @@ -129,11 +178,15 @@ jobs:
with:
python-version: "3.12"

- name: Install build dependencies
run: |
python3.12 -m pip install build

- name: Generate sdist (source distribution)
env:
BEZIER_NO_EXTENSION: "True"
run: |
python3.12 setup.py sdist
python3.12 -m build --sdist .

- uses: actions/upload-artifact@65462800fd760344b1a7b4382951275a0abb4808 # v4.3.3
with:
Expand Down Expand Up @@ -167,7 +220,7 @@ jobs:
"C:\ProgramData\chocolatey\lib\mingw\tools\install\mingw64\bin" >> $env:GITHUB_PATH

- name: Build wheels
uses: pypa/cibuildwheel@932529cab190fafca8c735a551657247fa8f8eaf #v2.19.1
uses: pypa/cibuildwheel@932529cab190fafca8c735a551657247fa8f8eaf # v2.19.1
env:
BEZIER_INSTALL_PREFIX: .\.nox\.cache\libbezier-release\usr
CIBW_ARCHS: AMD64
Expand All @@ -183,7 +236,7 @@ jobs:
--add-path .\.nox\.cache\libbezier-release\usr\bin
{wheel}
CIBW_TEST_COMMAND: pytest {project}/tests/unit
CIBW_TEST_REQUIRES: pytest
CIBW_TEST_REQUIRES: "pytest scipy sympy"

- uses: actions/upload-artifact@65462800fd760344b1a7b4382951275a0abb4808 # v4.3.3
with:
Expand Down
13 changes: 11 additions & 2 deletions noxfile.py
Original file line number Diff line number Diff line change
Expand Up @@ -667,8 +667,17 @@ def _cmake(session, build_type):
]
if IS_WINDOWS:
build_args.extend(["-G", "MinGW Makefiles"])
if IS_MACOS:
build_args.append("-DCMAKE_OSX_DEPLOYMENT_TARGET:STRING=13.0")

cmake_osx_deployment_target = os.environ.get("CMAKE_OSX_DEPLOYMENT_TARGET")
if cmake_osx_deployment_target is not None:
build_args.append(
f"-DCMAKE_OSX_DEPLOYMENT_TARGET={cmake_osx_deployment_target}"
)

cmake_fortran_compiler = os.environ.get("CMAKE_FORTRAN_COMPILER")
if cmake_fortran_compiler is not None:
build_args.append(f"-DCMAKE_Fortran_COMPILER={cmake_fortran_compiler}")

if os.environ.get("TARGET_NATIVE_ARCH") == "OFF":
build_args.append("-DTARGET_NATIVE_ARCH:BOOL=OFF")

Expand Down
30 changes: 0 additions & 30 deletions scripts/macos/build-wheels-arm.sh

This file was deleted.