pip fails continuously on slow connections

[ballo]

Description

Attempting to download a pip package via pip over a slow connection will reliably fail

Expected behavior

No response

pip version

24.3.1

Python version

3.13.0

OS

MacOS Ventura

How to Reproduce

1. Have a slow connection
2. Attempt to install a package that is over 1MB

Output

Downloading selenium-4.26.1-py3-none-any.whl (9.7 MB)
   ━━━━╺━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 1.0/9.7 MB 36.5 kB/s eta 0:03:57
ERROR: Exception:
Traceback (most recent call last):
  File "/Users/ball/pyenv/lib/python3.13/site-packages/pip/_vendor/urllib3/response.py", line 438, in _error_catcher
    yield
  File "/Users/ball/pyenv/lib/python3.13/site-packages/pip/_vendor/urllib3/response.py", line 561, in read
    data = self._fp_read(amt) if not fp_closed else b""
           ~~~~~~~~~~~~~^^^^^
  File "/Users/ball/pyenv/lib/python3.13/site-packages/pip/_vendor/urllib3/response.py", line 527, in _fp_read
    return self._fp.read(amt) if amt is not None else self._fp.read()
           ~~~~~~~~~~~~~^^^^^
  File "/Users/ball/pyenv/lib/python3.13/site-packages/pip/_vendor/cachecontrol/filewrapper.py", line 98, in read
    data: bytes = self.__fp.read(amt)
                  ~~~~~~~~~~~~~~^^^^^
  File "/opt/homebrew/Cellar/[email protected]/3.13.0_1/Frameworks/Python.framework/Versions/3.13/lib/python3.13/http/client.py", line 479, in read
    s = self.fp.read(amt)
  File "/opt/homebrew/Cellar/[email protected]/3.13.0_1/Frameworks/Python.framework/Versions/3.13/lib/python3.13/socket.py", line 719, in readinto
    return self._sock.recv_into(b)
           ~~~~~~~~~~~~~~~~~~~~^^^
  File "/opt/homebrew/Cellar/[email protected]/3.13.0_1/Frameworks/Python.framework/Versions/3.13/lib/python3.13/ssl.py", line 1304, in recv_into
    return self.read(nbytes, buffer)
           ~~~~~~~~~^^^^^^^^^^^^^^^^
  File "/opt/homebrew/Cellar/[email protected]/3.13.0_1/Frameworks/Python.framework/Versions/3.13/lib/python3.13/ssl.py", line 1138, in read
    return self._sslobj.read(len, buffer)
           ~~~~~~~~~~~~~~~~~^^^^^^^^^^^^^
TimeoutError: The read operation timed out

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "/Users/ball/pyenv/lib/python3.13/site-packages/pip/_internal/cli/base_command.py", line 105, in _run_wrapper
    status = _inner_run()
  File "/Users/ball/pyenv/lib/python3.13/site-packages/pip/_internal/cli/base_command.py", line 96, in _inner_run
    return self.run(options, args)
           ~~~~~~~~^^^^^^^^^^^^^^^
  File "/Users/ball/pyenv/lib/python3.13/site-packages/pip/_internal/cli/req_command.py", line 67, in wrapper
    return func(self, options, args)
  File "/Users/ball/pyenv/lib/python3.13/site-packages/pip/_internal/commands/install.py", line 379, in run
    requirement_set = resolver.resolve(
        reqs, check_supported_wheels=not options.target_dir
    )
  File "/Users/ball/pyenv/lib/python3.13/site-packages/pip/_internal/resolution/resolvelib/resolver.py", line 179, in resolve
    self.factory.preparer.prepare_linked_requirements_more(reqs)
    ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~^^^^^^
  File "/Users/ball/pyenv/lib/python3.13/site-packages/pip/_internal/operations/prepare.py", line 554, in prepare_linked_requirements_more
    self._complete_partial_requirements(
    ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~^
        partially_downloaded_reqs,
        ^^^^^^^^^^^^^^^^^^^^^^^^^^
        parallel_builds=parallel_builds,
        ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
    )
    ^
  File "/Users/ball/pyenv/lib/python3.13/site-packages/pip/_internal/operations/prepare.py", line 469, in _complete_partial_requirements
    for link, (filepath, _) in batch_download:
                               ^^^^^^^^^^^^^^
  File "/Users/ball/pyenv/lib/python3.13/site-packages/pip/_internal/network/download.py", line 184, in __call__
    for chunk in chunks:
                 ^^^^^^
  File "/Users/ball/pyenv/lib/python3.13/site-packages/pip/_internal/cli/progress_bars.py", line 55, in _rich_progress_bar
    for chunk in iterable:
                 ^^^^^^^^
  File "/Users/ball/pyenv/lib/python3.13/site-packages/pip/_internal/network/utils.py", line 65, in response_chunks
    for chunk in response.raw.stream(
                 ~~~~~~~~~~~~~~~~~~~^
        chunk_size,
        ^^^^^^^^^^^
    ...<22 lines>...
        decode_content=False,
        ^^^^^^^^^^^^^^^^^^^^^
    ):
    ^
  File "/Users/ball/pyenv/lib/python3.13/site-packages/pip/_vendor/urllib3/response.py", line 622, in stream
    data = self.read(amt=amt, decode_content=decode_content)
  File "/Users/ball/pyenv/lib/python3.13/site-packages/pip/_vendor/urllib3/response.py", line 560, in read
    with self._error_catcher():
         ~~~~~~~~~~~~~~~~~~~^^
  File "/opt/homebrew/Cellar/[email protected]/3.13.0_1/Frameworks/Python.framework/Versions/3.13/lib/python3.13/contextlib.py", line 162, in __exit__
    self.gen.throw(value)
    ~~~~~~~~~~~~~~^^^^^^^
  File "/Users/ball/pyenv/lib/python3.13/site-packages/pip/_vendor/urllib3/response.py", line 443, in _error_catcher
    raise ReadTimeoutError(self._pool, None, "Read timed out.")
pip._vendor.urllib3.exceptions.ReadTimeoutError: HTTPSConnectionPool(host='files.pythonhosted.org', port=443): Read timed out.

Code of Conduct

• I agree to follow the PSF Code of Conduct.

[notatallshaw]

Are you able to try increasing the timeout significantly to help? e.g. pip install --timeout 300 ...?

[ballo]

Well, I'm reporting a crash, so I'd say it's a bug regardless.

It's been a few days so I'm not on that connection any longer

[notatallshaw]

Well, I'm reporting a crash, so I'd say it's a bug regardless.

It's been a few days so I'm not on that connection any longer

The error you're receiving is a socket timeout:

  File "/opt/homebrew/Cellar/[email protected]/3.13.0_1/Frameworks/Python.framework/Versions/3.13/lib/python3.13/ssl.py", line 1138, in read
    return self._sslobj.read(len, buffer)
           ~~~~~~~~~~~~~~~~~^^^^^^^^^^^^^
TimeoutError: The read operation timed out

That's not a bug in pip, that's a network issue.

The default timeout is 15 seconds, you can either pass --timeout on each command, or you can configure it for every command by setting it in the config e.g. pip config set global.timeout 120.

[ballo]

It looks like the timeout is based not on incoming packets but the entire payload

[notatallshaw]

Not sure what you mean by payload, but the timeout isn't based on an entire file.

Pip uses requests, which uses urllib3 to download, and as per the stacktrace this was an ssl socket timeout. Could be there is some default chunk size the timeout is based on.