slim images are incompatible with manylinux2010+

[asottile-sentry]

notably they are missing libexpat.so.1 which is required by auditwheel

some example breakage:

$ docker run --rm -ti python:3.13.0-slim bash -c 'pip install -q pyuwsgi; uwsgi --help'
WARNING: Running pip as the 'root' user can result in broken permissions and conflicting behaviour with the system package manager, possibly rendering your system unusable.It is recommended to use a virtual environment instead: https://pip.pypa.io/warnings/venv. Use the --root-user-action option if you know what you are doing and want to suppress this warning.

[notice] A new release of pip is available: 24.2 -> 24.3.1
[notice] To update, run: pip install --upgrade pip
Traceback (most recent call last):
  File "/usr/local/bin/uwsgi", line 5, in <module>
    from pyuwsgi import run
ImportError: libexpat.so.1: cannot open shared object file: No such file or directory

#984 has some related discussion

[yosifkit]

libexpat.so.1 is not required by pypa/auditwheel. It is just on an allow list for users when building packages to distribute via pip so that they don't have to include a vendored version of the library in their package on pypi.org. Not every pip install is guaranteed to work in the slim images and users might need to install other packages via apt-get (or apk in Alpine) first.

python:<version>-slim

This image does not contain the common Debian packages contained in the default tag and only contains the minimal Debian packages needed to run python.

- Docker Hub description

#984 (comment):

While unfortunate, yes, this does mean you'll want/need to install libexpat1 manually in the slim images (it being included previously was incidental, not part of the intentional interface of the image).

[asottile-sentry]

I think you're incorrect / misguided with that -- if the soname is in the list for auditwheel then it is expected to be available on manylinux-compatible distributions

but if this is the stance then the images should at least mark that they are incompatible with manylinux such that they don't install incompatible distributions by default. one can do that by providing a _manylinux module in accordance with PEP 600

breaking manylinux compatibility is a really bad look for the official images

[LaurentGoderre]

@asottile-sentry requiring to install a dependency doesn't really make it incompatible. Slim images are for developer who want more control over what is in their image so they only install what is necessary.

[asottile-sentry]

that's not really the intention of the manylinux policy

[LaurentGoderre]

I don't see libexpat in the list of library for the manylinux policy. Furthermore, supporting that policy would force every user of slim into this which is the opposite of what the slim variant is. It might be better to propose a manylinux variant that to try to change the slim behavior.

[asottile-sentry]

that policy link is for manylinux1 (very old) -- the policies are now handled via auditwheel as of PEP 600

[yosifkit]

From PEP 0513

work on a stock CentOS 5.11 [...] system that contains the system package manager’s provided versions of these libraries

Nothing in the PEPs require that the system packages are installed to be manylinux compatible, just that the pip installed packages can link against them. The point of manylinux is that the creators of pip-installable packages create packages that work on many Linux distributions, not that a pip install works without preparation. Even PEP 513 explicitly listed out the Debian packages that provide the libraries that pip packages can link against; not an assumption that they are already installed on the system.

From the current policy: libX11.so.6, libXext.so.6, libXrender.so.1, libICE.so.6, libSM.so.6, libGL.so.1, libgobject-2.0.so.0, libgthread-2.0.so.0, and libglib-2.0.so.0 are also in the allow list but we don't see any reason to install X11 or graphical libraries in the images.

[asottile-sentry]

of course nothing requires one to be compatible -- but you previously were and are now not and your users are feeling the pain of that decision.

the goal is to make a pip install just work on many systems, and one of those is no longer the official python images seems pretty messed up to me

[yosifkit]

The -slim images have always been targeted as a minimal installation that has just enough to run python and where size-conscious users can control which extra libraries they install.

The non-slim images (like python:3.13.0 on Linux) are based on our buildpack-deps images which are designed such that "a majority of arbitrary gem install / npm install / pip install should be successful without additional header/development packages."

I'd still argue that both regular and slim images are manylinux-compatible, since any extra user-required system libraries are available for installation and at compatible versions. If I were to take a minimal Debian host (like a the debian:bookworm image) and apt-get install python3-pip, it too would not have all the .so files in the manylinux allow list, but you wouldn't call Debian's python + pip packages manylinux-incompatible.

[asottile-sentry]

?

$ docker run --rm -ti debian:bookworm bash -xc 'apt update -qq && apt install -y -qq --no-install-recommends python3-pip >& /dev/null && ls /lib/*/libexpat*'
+ apt update -qq
All packages are up to date.
+ apt install -y -qq --no-install-recommends python3-pip
+ ls /lib/aarch64-linux-gnu/libexpat.so.1 /lib/aarch64-linux-gnu/libexpat.so.1.8.10 /lib/aarch64-linux-gnu/libexpatw.so.1 /lib/aarch64-linux-gnu/libexpatw.so.1.8.10
/lib/aarch64-linux-gnu/libexpat.so.1
/lib/aarch64-linux-gnu/libexpat.so.1.8.10
/lib/aarch64-linux-gnu/libexpatw.so.1
/lib/aarch64-linux-gnu/libexpatw.so.1.8.10

[yosifkit]

Sure, it has the specific library that is disrupting your workflow today, but not every library in the lib_whitelist is installed (like libGL.so.1 or libICE.so.6):

    "lib_whitelist": ["libgcc_s.so.1", "libstdc++.so.6", "libm.so.6", "libdl.so.2", "librt.so.1", "libc.so.6", "libnsl.so.1", "libutil.so.1", "libpthread.so.0", "libX11.so.6", "libXext.so.6", "libXrender.so.1", "libICE.so.6", "libSM.so.6", "libGL.so.1", "libgobject-2.0.so.0", "libgthread-2.0.so.0", "libglib-2.0.so.0", "libresolv.so.2", "libz.so.1"],