Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Allow C extensions to declare compatibility with free-threading #116322

Closed
Tracked by #108219
swtaarrs opened this issue Mar 4, 2024 · 3 comments
Closed
Tracked by #108219

Allow C extensions to declare compatibility with free-threading #116322

swtaarrs opened this issue Mar 4, 2024 · 3 comments
Assignees
@swtaarrs
Labels
interpreter-core (Objects, Python, Grammar, and Parser dirs) topic-free-threading type-feature A feature request or enhancement

Comments

@swtaarrs
Copy link
Member

swtaarrs commented Mar 4, 2024 •
edited by bedevere-app bot
Loading

Feature or enhancement

There are a few pieces to this:

  1. Add a Py_mod_gil slot, as described in PEP 703, that multi-phase init modules can use to indicate that they support free-threading.
  2. Add a PyModule_ExperimentalSetGIL() function (discussed here) that single-phase init modules can use in place of Py_mod_gil.
  3. Mark all built-in modules as free-threading compatible with one of the above mechanisms.
  4. Enable the GIL while loading a C module and leave it permanently enabled if the module does not declare free-threading compatibility.

1-3 are addressed in gh-116882; 4 will be addressed in a separate PR.

Linked PRs
@swtaarrs swtaarrs added type-feature A feature request or enhancement interpreter-core (Objects, Python, Grammar, and Parser dirs) topic-free-threading labels Mar 4, 2024
This was referenced Mar 4, 2024
Open
Merged
@swtaarrs swtaarrs self-assigned this Mar 11, 2024
@swtaarrs
Copy link
Member Author

What warning category should this issue? ImportWarning sounds like a good fit at first, but it's ignored by the default warning filters, which we don't want. Of the categories not ignored by default, RuntimeWarning seems like the best fit to me.

This was referenced Mar 13, 2024
Open
Merged
@swtaarrs swtaarrs mentioned this issue Apr 3, 2024
Closed
@swtaarrs swtaarrs changed the title Implement Py_mod_gil slot for free-threading-safe extensions Allow C extensions to declare compatibility with free-threading May 2, 2024
colesbury pushed a commit that referenced this issue May 3, 2024
@swtaarrs
gh-116322: Add Py_mod_gil module slot (#116882)
c2627d6 
This PR adds the ability to enable the GIL if it was disabled at
interpreter startup, and modifies the multi-phase module initialization
path to enable the GIL when loading a module, unless that module's spec
includes a slot indicating it can run safely without the GIL.

PEP 703 called the constant for the slot `Py_mod_gil_not_used`; I went
with `Py_MOD_GIL_NOT_USED` for consistency with gh-104148.

A warning will be issued up to once per interpreter for the first
GIL-using module that is loaded. If `-v` is given, a shorter message
will be printed to stderr every time a GIL-using module is loaded
(including the first one that issues a warning).
@swtaarrs swtaarrs mentioned this issue May 3, 2024
Merged
encukou added a commit to encukou/cpython that referenced this issue May 6, 2024
@encukou
pythongh-116322: Rename PyModule_ExperimentalSetGIL to PyUnstable_Mod…
82f3c58 
…ule_SetGIL
@bedevere-app bedevere-app bot mentioned this issue May 6, 2024
Merged
encukou added a commit that referenced this issue May 6, 2024
@encukou
gh-116322: Rename PyModule_ExperimentalSetGIL to PyUnstable_Module_Se…
417dd3a 
…tGIL (GH-118645)
colesbury pushed a commit that referenced this issue May 7, 2024
@swtaarrs
gh-116322: Enable the GIL while loading C extension modules (#118560)
853163d 
Add the ability to enable/disable the GIL at runtime, and use that in
the C module loading code.

We can't know before running a module init function if it supports
free-threading, so the GIL is temporarily enabled before doing so. If
the module declares support for running without the GIL, the GIL is
later disabled. Otherwise, the GIL is permanently enabled, and will
never be disabled again for the life of the current interpreter.
@colesbury
Copy link
Contributor

@swtaarrs is this finished now?

@swtaarrs
Copy link
Member Author

swtaarrs commented May 7, 2024

Yep!

@swtaarrs swtaarrs closed this as completed May 7, 2024
SonicField pushed a commit to SonicField/cpython that referenced this issue May 8, 2024
@swtaarrs @SonicField
pythongh-116322: Add Py_mod_gil module slot (python#116882)
f301715 
This PR adds the ability to enable the GIL if it was disabled at
interpreter startup, and modifies the multi-phase module initialization
path to enable the GIL when loading a module, unless that module's spec
includes a slot indicating it can run safely without the GIL.

PEP 703 called the constant for the slot `Py_mod_gil_not_used`; I went
with `Py_MOD_GIL_NOT_USED` for consistency with pythongh-104148.

A warning will be issued up to once per interpreter for the first
GIL-using module that is loaded. If `-v` is given, a shorter message
will be printed to stderr every time a GIL-using module is loaded
(including the first one that issues a warning).
SonicField pushed a commit to SonicField/cpython that referenced this issue May 8, 2024
@encukou @SonicField
pythongh-116322: Rename PyModule_ExperimentalSetGIL to PyUnstable_Mod…
7678b39 
…ule_SetGIL (pythonGH-118645)
SonicField pushed a commit to SonicField/cpython that referenced this issue May 8, 2024
@swtaarrs @SonicField
pythongh-116322: Enable the GIL while loading C extension modules (py…
0d76d9c 
…thon#118560)

Add the ability to enable/disable the GIL at runtime, and use that in
the C module loading code.

We can't know before running a module init function if it supports
free-threading, so the GIL is temporarily enabled before doing so. If
the module declares support for running without the GIL, the GIL is
later disabled. Otherwise, the GIL is permanently enabled, and will
never be disabled again for the life of the current interpreter.
@zmedico zmedico mentioned this issue Jun 16, 2024
Merged
@bedevere-app bedevere-app bot mentioned this issue Jul 25, 2024
Merged
colesbury added a commit to colesbury/cpython that referenced this issue Jul 25, 2024
@colesbury
pythongh-116322: Fix typo in moduleobject.c
1aebb36
@bedevere-app bedevere-app bot mentioned this issue Jul 25, 2024
Closed
kumaraditya303 pushed a commit that referenced this issue Jul 25, 2024
@serhiy-storchaka
gh-116322: Fix typo in the #ifdef check (#122268)
9bb2e46
miss-islington pushed a commit to miss-islington/cpython that referenced this issue Jul 25, 2024
@serhiy-storchaka @miss-islington
pythongh-116322: Fix typo in the GH-ifdef check (pythonGH-122268)
f2060f1 
(cherry picked from commit 9bb2e46)

Co-authored-by: Serhiy Storchaka <[email protected]>
AlexWaygood added a commit to colesbury/cpython that referenced this issue Jul 25, 2024
@AlexWaygood
Merge branch 'main' into pythongh-116322-typo
47d9618
@bedevere-app bedevere-app bot mentioned this issue Jul 25, 2024
Merged
kumaraditya303 pushed a commit that referenced this issue Jul 25, 2024
@miss-islington @serhiy-storchaka
[3.13] gh-116322: Fix typo in the GH-ifdef check (GH-122268) (#122284)
88a96fc 
gh-116322: Fix typo in the GH-ifdef check (GH-122268)
(cherry picked from commit 9bb2e46)

Co-authored-by: Serhiy Storchaka <[email protected]>
stefanholek added a commit to stefanholek/rl that referenced this issue Oct 14, 2024
@stefanholek
Mark readline module in no-gil builds.
65457c6 
This means: "Do not re-enable the GIL when importing rl.readline."

See python/cpython#116322
See python/cpython#116882
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees

@swtaarrs swtaarrs

Labels
interpreter-core (Objects, Python, Grammar, and Parser dirs) topic-free-threading type-feature A feature request or enhancement
Projects
None yet
Milestone
No milestone
Development

No branches or pull requests
2 participants
@swtaarrs @colesbury