Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Update most dependencies to latest (2024-06) #318

Merged
merged 26 commits into from
Jun 18, 2024
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
26 commits
Select commit Hold shift + click to select a range
3be79b2
`session.run()` can no longer pass kwargs to callables.
dhermes Jun 17, 2024
d1ebe7a
Replace `run_always()` with alias `run_install()`.
dhermes Jun 17, 2024
23cf406
`session.run()` can no longer pass kwargs to callables (another insta…
dhermes Jun 17, 2024
7d83c8a
Ran `nox -s update_generated`.
dhermes Jun 17, 2024
245f6fd
Update lower bound for `Cython` dependency.
dhermes Jun 17, 2024
ee15816
Not ready for Numpy 2.0 yet (breaks Cython in ways I have yet to sort…
dhermes Jun 17, 2024
b0fe44c
Update all lower bounds.
dhermes Jun 17, 2024
19656b7
Ran `nox -s blacken` with updated deps.
dhermes Jun 17, 2024
619a6ef
Update dependencies in `docs/requirements.txt` and `setup.py`.
dhermes Jun 17, 2024
6c76aba
Update `dumpbin` output on Windows doctest.
dhermes Jun 17, 2024
63cda4d
Put NumPy upper bound on `build-wheel-for-doctest.sh`.
dhermes Jun 17, 2024
dcf10cb
Fix `readelf` Linux output.
dhermes Jun 17, 2024
278d11d
Limit to x86 for macOS (CI machines in GitHub Actions are now multi-a…
dhermes Jun 17, 2024
5344cb6
Try the actual arch (still think this will not work).
dhermes Jun 18, 2024
184ab35
Try to specify `architecture` in `actions/setup-python`.
dhermes Jun 18, 2024
7459590
Try using `CMAKE_OSX_ARCHITECTURES`.
dhermes Jun 18, 2024
36082f5
Downgrade to `macos-13`.
dhermes Jun 18, 2024
5317aa0
Remove the `--require-archs`.
dhermes Jun 18, 2024
51bbcf6
Use `session.run()` to delete directories.
dhermes Jun 18, 2024
faac2a4
Add debug `ls -alFG`.
dhermes Jun 18, 2024
5071dbb
Temporarily disable workflow runs not needed.
dhermes Jun 18, 2024
e2311fe
Run even less, forgot `external=True`.
dhermes Jun 18, 2024
55595a4
Is the issue `pip` vs. `python -m pip`?
dhermes Jun 18, 2024
0dd613c
Some more debug.
dhermes Jun 18, 2024
23da3a9
Target 13.0 (not 13.6).
dhermes Jun 18, 2024
2889190
Revert debug changes.
dhermes Jun 18, 2024
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
2 changes: 1 addition & 1 deletion .github/workflows/macos.yaml
Original file line number Diff line number Diff line change
Expand Up @@ -30,7 +30,7 @@ on:

jobs:
ci:
runs-on: macos-latest
runs-on: macos-13
Copy link
Owner Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This is temporary!

See: actions/setup-python#547

In particular, the architecture on macos-latest (macos-14) machines means that mixed-architecture dylibs are produced by gcc.

See for example a failed build:

...
Successfully built bezier
nox > delocate-wheel --wheel-dir /var/folders/dm/88b38gj92jj53dgxdsm12qf00000gn/T/tmpraciqawj --verbose /var/folders/dm/88b38gj92jj53dgxdsm12qf00000gn/T/tmpizrl0uv3/bezier-2023.7.29.dev1-cp311-cp311-macosx_10_9_universal2.whl
Fixing: /var/folders/dm/88b38gj92jj53dgxdsm12qf00000gn/T/tmpizrl0uv3/bezier-2023.7.29.dev1-cp311-cp311-macosx_10_9_universal2.whl
Traceback (most recent call last):
  File "/Users/runner/work/bezier/bezier/.nox/doctest/bin/delocate-wheel", line 8, in <module>
    sys.exit(main())
             ^^^^^^
  File "/Users/runner/work/bezier/bezier/.nox/doctest/lib/python3.11/site-packages/delocate/cmd/delocate_wheel.py", line 110, in main
    copied = delocate_wheel(
             ^^^^^^^^^^^^^^^
  File "/Users/runner/work/bezier/bezier/.nox/doctest/lib/python3.11/site-packages/delocate/delocating.py", line 971, in delocate_wheel
    copied_libs = delocate_path(
                  ^^^^^^^^^^^^^^
  File "/Users/runner/work/bezier/bezier/.nox/doctest/lib/python3.11/site-packages/delocate/delocating.py", line 509, in delocate_path
    lib_dict = tree_libs_from_directory(
               ^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/Users/runner/work/bezier/bezier/.nox/doctest/lib/python3.11/site-packages/delocate/libsana.py", line 377, in tree_libs_from_directory
    return _tree_libs_from_libraries(
           ^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/Users/runner/work/bezier/bezier/.nox/doctest/lib/python3.11/site-packages/delocate/libsana.py", line 304, in _tree_libs_from_libraries
    for depending_path, install_name in get_dependencies(
  File "/Users/runner/work/bezier/bezier/.nox/doctest/lib/python3.11/site-packages/delocate/libsana.py", line 102, in get_dependencies
    for install_name in get_install_names(lib_fname):
                        ^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/Users/runner/work/bezier/bezier/.nox/doctest/lib/python3.11/site-packages/delocate/tools.py", line 559, in get_install_names
    names_data = _check_ignore_archs(_parse_otool_install_names(otool.stdout))
                 ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/Users/runner/work/bezier/bezier/.nox/doctest/lib/python3.11/site-packages/delocate/tools.py", line 413, in _check_ignore_archs
    raise NotImplementedError(
NotImplementedError: This function does not support separate values per-architecture: {'x86_64': [('/usr/lib/libSystem.B.dylib', '1.0.0', '1336.0.0')], 'arm64': [('@rpath/libbezier.2023.dylib', '2023.0.0', '2023.7.28'), ('/usr/lib/libSystem.B.dylib', '1.0.0', '1336.0.0')]}
nox > Command delocate-wheel --wheel-dir /var/folders/dm/88b38gj92jj53dgxdsm12qf00000gn/T/tmpraciqawj --verbose /var/folders/dm/88b38gj92jj53dgxdsm12qf00000gn/T/tmpizrl0uv3/bezier-2023.7.29.dev1-cp311-cp311-macosx_10_9_universal2.whl failed with exit code 1
...

strategy:
matrix:
python-version:
Expand Down
13 changes: 10 additions & 3 deletions docs/python/binary-extension.rst
Original file line number Diff line number Diff line change
Expand Up @@ -107,14 +107,14 @@ The ``bezier._speedup`` module depends on this local copy of ``libbezier``:

$ readelf -d _speedup.cpython-311-x86_64-linux-gnu.so

Dynamic section at offset 0x498000 contains 27 entries:
Dynamic section at offset 0x49c000 contains 27 entries:
Tag Type Name/Value
0x000000000000000f (RPATH) Library rpath: [$ORIGIN/../bezier.libs]
0x0000000000000001 (NEEDED) Shared library: [libbezier-631d8eda.so.2023.7.28]
0x0000000000000001 (NEEDED) Shared library: [libpthread.so.0]
0x0000000000000001 (NEEDED) Shared library: [libc.so.6]
0x000000000000000c (INIT) 0x6000
0x000000000000000d (FINI) 0x7ef80
0x000000000000000d (FINI) 0x7f630
Copy link
Owner Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Expected when Cython generated code gets updated

...

and the local copy of ``libbezier`` depends on the other dependencies in
Expand Down Expand Up @@ -419,8 +419,15 @@ and the local copy of ``libbezier`` depends on the other dependencies in

libgcc_s_seh-1-5c71c85c0ca01174917203266ba98140.dll
libgfortran-5-08073c6868a1df2cbc5609e49cbe3ad8.dll
api-ms-win-crt-environment-l1-1-0.dll
api-ms-win-crt-heap-l1-1-0.dll
api-ms-win-crt-math-l1-1-0.dll
api-ms-win-crt-private-l1-1-0.dll
api-ms-win-crt-runtime-l1-1-0.dll
api-ms-win-crt-stdio-l1-1-0.dll
api-ms-win-crt-string-l1-1-0.dll
api-ms-win-crt-time-l1-1-0.dll
KERNEL32.dll
msvcrt.dll

Summary
...
Expand Down
6 changes: 3 additions & 3 deletions docs/requirements.txt
Original file line number Diff line number Diff line change
@@ -1,8 +1,8 @@
# NumPy is required for building `bezier` from source.
numpy >= 1.25.2
numpy >= 1.26.4, < 2
# Sphinx and related are required for building documentation.
Sphinx >= 6.2.1
Sphinx >= 7.3.7
# See: https://github.com/readthedocs/sphinx_rtd_theme/issues/1463
sphinx-copybutton >= 0.5.2
sphinx-docstring-typing >= 0.0.4
sphinx-rtd-theme >= 1.2.2
sphinx-rtd-theme >= 2.0.0
75 changes: 50 additions & 25 deletions noxfile.py
Original file line number Diff line number Diff line change
Expand Up @@ -10,6 +10,7 @@
# See the License for the specific language governing permissions and
# limitations under the License.

import functools
import glob
import os
import pathlib
Expand All @@ -28,29 +29,29 @@
IS_MACOS = sys.platform == "darwin"
IS_WINDOWS = os.name == "nt"
DEPS = {
"black": "black >= 23.7.0",
"black": "black >= 24.4.2",
"cmake-format": "cmake-format >= 0.6.13",
"cmake": "cmake >= 3.27.0",
"cmake": "cmake >= 3.29.5.1",
"coverage": "coverage",
"Cython": "Cython >= 3.0.2",
"delocate": "delocate >= 0.10.4",
"delvewheel": "delvewheel >= 1.4.0",
"Cython": "Cython >= 3.0.10",
"delocate": "delocate >= 0.11.0",
"delvewheel": "delvewheel >= 1.6.0",
"docutils": "docutils",
"flake8": "flake8",
"flake8-import-order": "flake8-import-order",
"jsonschema": "jsonschema >= 4.18.4",
"jsonschema": "jsonschema >= 4.22.0",
"lcov-cobertura": "lcov-cobertura >= 2.0.2",
"matplotlib": "matplotlib >= 3.7.2",
"numpy": "numpy >= 1.25.2",
"pycobertura": "pycobertura >= 3.2.1",
"matplotlib": "matplotlib >= 3.9.0",
"numpy": "numpy >= 1.26.4, < 2",
Copy link
Owner Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Handling 2.0 in another PR

"pycobertura": "pycobertura >= 3.3.2",
"Pygments": "Pygments",
"pylint": "pylint >= 2.17.5",
"pytest": "pytest >= 7.4.0",
"pylint": "pylint >= 3.2.3",
"pytest": "pytest >= 8.2.2",
"pytest-cov": "pytest-cov",
"referencing": "referencing >= 0.30.0",
"scipy": "scipy >= 1.11.1",
"sympy": "sympy >= 1.12",
"seaborn": "seaborn >= 0.12.2",
"referencing": "referencing >= 0.35.1",
"scipy": "scipy >= 1.13.1",
"sympy": "sympy >= 1.12.1",
"seaborn": "seaborn >= 0.13.2",
}
BASE_DEPS = (DEPS["numpy"], DEPS["pytest"])
NOX_DIR = os.path.abspath(os.path.dirname(__file__))
Expand All @@ -67,6 +68,10 @@
RELEASE_SESSION_NAME = "libbezier-release"
INSTALL_PREFIX_ENV = "BEZIER_INSTALL_PREFIX"
EXTRA_DLL_ENV = "BEZIER_EXTRA_DLL"
_OS_MAKEDIRS_EXIST_OK = functools.partial(os.makedirs, exist_ok=True)
Copy link
Owner Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Session.run() has deprecated the use of kwargs passing through to callables

_SHUTIL_RMTREE_IGNORE_ERRORS = functools.partial(
shutil.rmtree, ignore_errors=True
)


def get_path(*names):
Expand Down Expand Up @@ -286,6 +291,8 @@ def _macos_doctest_install(session, install_prefix):
# 2. Build the wheel from source.
basic_dir = tempfile.mkdtemp()
session.run(
"python",
"-m",
Comment on lines +294 to +295
Copy link
Owner Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I added this when debugging (I was worried something was wrong in pip resolution). I didn't end up needing this change but python -m pip is more canonical so keeping it.

"pip",
"wheel",
".",
Expand All @@ -307,7 +314,14 @@ def _macos_doctest_install(session, install_prefix):
)
# 4. Install from the repaired wheel.
session.run(
"pip", "install", "bezier", "--no-index", "--find-links", repaired_dir
"python",
"-m",
"pip",
"install",
"bezier",
"--no-index",
"--find-links",
repaired_dir,
)
# 5. Clean up temporary directories.
shutil.rmtree(basic_dir, ignore_errors=True)
Expand All @@ -320,6 +334,8 @@ def _windows_doctest_install(session, install_prefix):
# 2. Build the wheel from source.
basic_dir = tempfile.mkdtemp()
session.run(
"python",
"-m",
"pip",
"wheel",
".",
Expand All @@ -342,7 +358,14 @@ def _windows_doctest_install(session, install_prefix):
)
# 4. Install from the repaired wheel.
session.run(
"pip", "install", "bezier", "--no-index", "--find-links", repaired_dir
"python",
"-m",
"pip",
"install",
"bezier",
"--no-index",
"--find-links",
repaired_dir,
)
# 5. Clean up temporary directories.
shutil.rmtree(basic_dir, ignore_errors=True)
Expand Down Expand Up @@ -592,7 +615,7 @@ def _cmake_libbezier_root(session, build_type):
relative_path = session.cache_dir / build_session_name
# Convert to an absolute path.
libbezier_root = get_path(relative_path)
session.run(os.makedirs, libbezier_root, exist_ok=True)
session.run(_OS_MAKEDIRS_EXIST_OK, libbezier_root)
return libbezier_root


Expand Down Expand Up @@ -627,13 +650,13 @@ def _cmake(session, build_type):
session.install(DEPS["cmake"])
cmake_external = False
else:
session.run_always(print, "Using pre-installed ``cmake``")
session.run_always("cmake", "--version", external=cmake_external)
session.run_install(print, "Using pre-installed ``cmake``")
session.run_install("cmake", "--version", external=cmake_external)
Copy link
Owner Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

run_always() is now just an alias for run_install()


# Prepare build and install directories.
build_dir = os.path.join(libbezier_root, "build")
install_prefix = os.path.join(libbezier_root, "usr")
session.run_always(os.makedirs, build_dir, exist_ok=True)
session.run_install(_OS_MAKEDIRS_EXIST_OK, build_dir)

# Run ``cmake`` to prepare for / configure the build.
build_args = [
Expand All @@ -644,14 +667,16 @@ def _cmake(session, build_type):
]
if IS_WINDOWS:
build_args.extend(["-G", "MinGW Makefiles"])
if IS_MACOS:
build_args.append("-DCMAKE_OSX_DEPLOYMENT_TARGET:STRING=13.0")
Comment on lines +670 to +671
Copy link
Owner Author

@dhermes dhermes Jun 18, 2024

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This is temporary! The default value used by CMake is whatever the current version is on the machine. Here that's 13.6 as of https://github.com/actions/runner-images/blob/3793317199f7f1eebff21ae96c77878546f06389/images/macos/macos-13-Readme.md.

However, only 13.0 is allowed, e.g.:

$ python3.11 -m pip debug --verbose | grep cp311-cp311-macosx_13
  cp311-cp311-macosx_13_0_x86_64
  cp311-cp311-macosx_13_0_intel
  cp311-cp311-macosx_13_0_fat64
  cp311-cp311-macosx_13_0_fat32
  cp311-cp311-macosx_13_0_universal2
  cp311-cp311-macosx_13_0_universal

if os.environ.get("TARGET_NATIVE_ARCH") == "OFF":
build_args.append("-DTARGET_NATIVE_ARCH:BOOL=OFF")

build_args.extend(["-S", os.path.join("src", "fortran"), "-B", build_dir])
session.run_always(*build_args, external=cmake_external)
session.run_install(*build_args, external=cmake_external)

# Build and install.
session.run_always(
session.run_install(
"cmake",
"--build",
build_dir,
Expand All @@ -663,7 +688,7 @@ def _cmake(session, build_type):
)

# Get information on how the build was configured.
session.run_always("cmake", "-L", build_dir, external=cmake_external)
session.run_install("cmake", "-L", build_dir, external=cmake_external)

return install_prefix

Expand Down Expand Up @@ -722,7 +747,7 @@ def clean(session):
get_path("tests", "unit", "*.pyc"),
)
for dir_path in clean_dirs:
session.run(shutil.rmtree, dir_path, ignore_errors=True)
session.run(_SHUTIL_RMTREE_IGNORE_ERRORS, dir_path)
for glob_path in clean_globs:
for filename in glob.glob(glob_path):
session.run(os.remove, filename)
2 changes: 1 addition & 1 deletion scripts/manylinux/build-wheel-for-doctest.sh
Original file line number Diff line number Diff line change
Expand Up @@ -31,7 +31,7 @@ fi

# 0. Install the Python dependencies
"${PY_ROOT}/bin/python" -m pip install --upgrade pip
"${PY_ROOT}/bin/python" -m pip install --upgrade auditwheel cmake nox numpy
"${PY_ROOT}/bin/python" -m pip install --upgrade auditwheel cmake nox 'numpy >= 1.26.4, < 2'

# 1. Make sure no previous build artifacts are still around
cd "${BEZIER_ROOT}"
Expand Down
2 changes: 1 addition & 1 deletion setup.py
Original file line number Diff line number Diff line change
Expand Up @@ -72,7 +72,7 @@
""".format(
install_prefix=INSTALL_PREFIX_ENV, no_extension=NO_EXTENSION_ENV
)
REQUIREMENTS = ("numpy >= 1.25.2",)
REQUIREMENTS = ("numpy >= 1.26.4, < 2",)
# See: https://www.python.org/dev/peps/pep-0508/
# Dependency specification for Python Software Packages
EXTRAS_REQUIRE = {
Expand Down
Loading
Loading