Status of packaging tools

[rgommers]

• packaging: 24.0 has full support
• Build frontends:
  • pip: in 24.1 (released 21 June)- Upgrade vendored packaging lib pypa/pip#12300
  • build: probably nothing to do, since it depends on latest packaging
• Build backends:
  • meson-python: done modulo adding CI job, see meson-python#499
  • scikit-build-core: fix: free threaded Python scikit-build/scikit-build-core#741
  • setuptools: 69.5.0 upgraded the vendored packaging library to 24.0
• manylinux: feat: add CPython 3.13 --disable-gil build pypa/manylinux#1564
• cibuildwheel: [request] support --disable-gil builds (PEP703) pypa/cibuildwheel#1657 (no macOS installer yet, waiting for 3.13.0b2)
• wheel publishing tools (probably all fine, will be ticked off once they're known to work):
  • auditwheel
  • delvewheel
  • delocate
  • repair-wheel

[henryiii]

• Scikit-build-core done in fix: free threaded Python scikit-build/scikit-build-core#741
• manylinux done
• Cibuildwheel done except no macOS yet, waiting on beta 2 for official installers

[rgommers]

Thanks @henryiii! Table updated.

[henryiii]

Looks like both meson and CMake don't support Windows + free-threading yet, they don't know to look for the library with the "t" for linking. I think I can fix this in scikit-build-core if I can tell it the correct library from sysconfig info.

[rgommers]

Yeah, working on the same thing for Meson. I think adding the t in python313t.lib based on:

bool(sysconfig.get_config_var('Py_GIL_DISABLED'))

should do the job. Plus the debug flavor of it (there's no limited API variant).

[henryiii]

Windows free-threaded fixed in scikit-build-core 0.9.5.

[henryiii]

FYI, sysconfig's LIBRARY is correct as far as the "t" goes but is .dll instead of .lib.

[henryiii]

Free-threading including macOS with 3.13.0b2 fully supported in cibuildwheel 2.19.0.

[rgommers]

The final release of Pip 24.1 is out 🎉. That was the biggest missing piece here, we're in a pretty happy place now.

[rgommers]

The Windows support in Meson by fixed in mesonbuild/meson#13338. @henryiii there is a second issue on Windows beyond libpython, namely that pyconfig.h is missing the Py_GIL_DISABLED define (discussed in https://discuss.python.org/t/windows-installer-freethreading-and-building-extension-modules/54391/2). I only lightly browsed your CMake/scikit-build-core PRs, so maybe I missed it - but it wasn't clear to me that that fix was included.

EDIT: xref https://gitlab.kitware.com/cmake/cmake/-/issues/26016 for CMake Windows issue.

[minrk]

Is it appropriate to add tracking of pyO3 and cffi here in build backends?

• pyO3 tracking issue: Tracking issue for no-gil/freethreaded work PyO3/pyo3#4265
• cffi: Support for new cp313t GIL-free ABI variant python-cffi/cffi#40

I was testing Jupyter today, and of the extension dependencies for JupyterLab, these appear to both be limiting factors. Most extensions are working fine, though:

• pyzmq works without modification when built with the cython prerelease
• argon-cffi-bindings uses cffi which unconditionally sets the unsupported py_limited_api=True (I got this to build with limited api is not available in free-threaded cpython python-cffi/cffi#104)
• jsonschema uses rpds-py, which uses pyO3. rpds builds, but crashes at import

so I can run a nogil kernel, but not the UI yet.

[rgommers]

Hey @minrk, thanks for the input! I'd say that these two aren't packaging tools so this isn't quite the right issue - and they're both important enough to deserve their own tracking issue.

PyO3 working will take a while, work has just started and it's quite complex. @ngoldbaum perhaps you could open a separate tracking issue for this?

cffi hasn't moved since December I think (aside from the PR you just opened), and it's the first time it has come up here. Maybe you want to open a new issue for it on this repo?

[ngoldbaum]

I opened #53 for PyO3, thanks for the reminder.

[minrk]

Thanks, I somehow overlooked that pybind11 is not in this list, which is what I was thinking of as a peer for these. Opened #54 for cffi.

[henryiii]

but it wasn't clear to me that that fix was included.

FYI, this was fixed in CMake 3.30.3 a month or two ago.

[rgommers]

Great, thanks @henryiii!

[rgommers]

Windows wheels are also starting to materialize and that all looks good, so I ticked off delvewheel. I'm sure there are some corner case bugs lurking somewhere, but overall we're in good shape here.