Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Avoid using internal functions/object with multiprocessing #843

Merged
merged 1 commit into from
Sep 22, 2023
Merged

Avoid using internal functions/object with multiprocessing #843

merged 1 commit into from
Sep 22, 2023

Commits on Sep 12, 2023

  1. Avoid using internal functions/object with multiprocessing 

    A Multiprocessing start method is the technique used
by the Multiprocessing library to start child processes in Python.

There are three start methods:

* spawn: start a new Python process.
* fork: copy a Python process from an existing process.
* forkserver: new process from which future forked processes will be copied.

Each OS has a list of supported start methods:
* Windows (win32): spawn
* MacOS (darwin): spawn (default), fork, forkserver.
* Linux (unix): spawn, fork (default), forkserver.

Spawn not only is the most supported, but as per recent investigations
of the devs of the Multiprocessing library, is the most secure on all
the OS.

Spawn method makes use of the Pickle library for starting the new
subprocesses and Pickle is unable to process internal objects and
functions.

Get rid an internal function (handler inside the make_logging_handler
method of the Command class) and use a callable object instead.

Consider substituting internal objects and functions with Pickle
compliant options all around the code and eventually enforce the usage
of the spawn method.

Signed-off-by: Giulio Calacoci <[email protected]>
    @gcalacoci
    gcalacoci committed Sep 12, 2023
    Configuration menu
    Copy the full SHA
    0197300 View commit details
    Browse the repository at this point in the history