AttributeError: module 'collections' has no attribute 'Callable' with python-dateutil<2.7 for Python >= 3.10

[pcjedi]

Describe the bug

While investigating issue #3338, it was discovered that running the botocore test suite with python-dateutil<2.7 causes multiple test failures, but only when using Python versions >= 3.10. This suggests that botocore relies on behavior or bug fixes in python-dateutil that are only present in python-dateutil>=2.7, and the issue appears specifically when using newer Python versions.

Regression Issue

• Select this option if this issue appears to be a regression.

Expected Behavior

The botocore test suite should pass with all supported versions of python-dateutil as per the version constraints specified in its dependencies, and the tests should be compatible with Python >= 3.10.

Current Behavior

When testing botocore with python-dateutil<2.7 and Python >= 3.10, multiple test cases fail. For example, running tests with python-dateutil==2.6.1 results in failures such as:

FAILED unit/test_credentials.py::TestAssumeRoleCredentialProvider::test_assume_role_populates_session_name_on_refresh - AttributeError: module 'collections' has no attribute 'Callable'

a complete list can be seen at
https://github.com/pcjedi/botocore/actions/runs/12727041761/job/35476098457

This behavior suggests that botocore inadvertently depends on features or bug fixes that are only present in python-dateutil>=2.7.

Reproduction Steps

To reproduce this issue, use the following script:

git clone https://github.com/boto/botocore.git
cd botocore
python -m venv .venv
.venv/bin/pip install . "python-dateutil==2.6.1" -r requirements.txt
.venv/bin/pytest 

This will install botocore along with python-dateutil==2.6.1 and run the test suite, which will fail with Python >= 3.10

Possible Solution

Update the version constraint for python-dateutil in botocore to >=2.7. Testing has shown that all botocore tests pass successfully with python-dateutil>=2.7.

https://github.com/pcjedi/botocore/actions/runs/12726749722

Additional Information/Context

This issue specifically affects testing with Python >= 3.10 and python-dateutil<2.7

SDK version used

1.36.0

Environment details (OS name and version, etc.)

debian

[nateprewitt]

Hi @pcjedi, as we can see in the stacktrace you linked:

     File "/opt/hostedtoolcache/Python/3.10.16/x64/lib/python3.10/site-packages/dateutil/parser.py", line 1182, in parse
      return DEFAULTPARSER.parse(timestr, **kwargs)
    File "/opt/hostedtoolcache/Python/3.10.16/x64/lib/python3.10/site-packages/dateutil/parser.py", line 587, in parse
      if (isinstance(tzinfos, collections.Callable) or
  AttributeError: module 'collections' has no attribute 'Callable'

This error is coming from python-dateutil itself not from botocore. The issue you're encountering is because python-dateutil does not support Python 3.10 prior to 2.9 as noted in their metadata. This isn't a detail we intend to manage for upstream libraries. It adds more complexity to dependencies between projects and is likely to have unintended ripple effects.

[pcjedi]

Hi @nateprewitt ,
it is fixed with dateutil/dateutil#612, which is part of python-dateutil 2.7.0 which was released more than 6 years ago.
with the second link i provided, you can see that with python-dateutil==2.7.0 all test succeed.

[pcjedi]

This issue is not limited to testing, but can also occur in production when using python >= 3.10

The specific error:

AttributeError: module 'collections' has no attribute 'Callable'

can be reproduced with the following steps:

pip install boto3 "python-dateutil<2.7"
python -c "import boto3; boto3.client('s3').list_buckets()"

This demonstrates that botocore is incompatible with python-dateutil<2.7 when using Python >= 3.10.
merging #3340 would fix this issue, together with #3338

adding
python-dateutil>=2.7; python_version>="3.10" would also work, but adds a little more complexity

[nateprewitt]

@pcjedi to be clear, everything is incompatible with python-dateutil in Python>=3.10, because the library explicitly does not support Python 3.8 or later. That is not an issue with botocore.

As stated above, we won't generally micromanage other libraries version requirements. Especially if the failures are not related to our code. Looking back, python-dateutil should have used python_requires but adoption was still fairly low at that point. I don't think we intend to go back and address all of their version splits. We'd recommend users upgrade to the latest releases but otherwise leave those decisions to the users of the library.

[pcjedi]

@nateprewitt on these 3 things i have remarks:

[python-dateutil] explicitly does not support Python 3.8

python-dateutil supports python >=2.7, !=3.0.*, !=3.1.*, !=3.2.* via python_requires

failures are not related to our code

botocore claims to be working with python-dateutil>=2.1. You can fix this by:

• Raise python-dateutil Constraint to >=2.7 #3340

upgrade to the latest releases

Let me summarize for whom this issue is of concern. maintainers of python code/packages, that:

• depend on boto3/botocore
• not depend on python-dateutil, not even indirectly via e.g. pandas>=1.1 or matplotlib>=3.4 which provide working dependency requirements regarding python-dateutil
• test with varying dependency resolutions, e.g. using uv lowest resolution (botocore is doing something similar regarding urllib3 and awscrt)

these packages might be private due to corporate interests, like in my case. But i have provided a minimal example

[RyanFitzSimmonsAK]

Hi @pcjedi, thanks for reaching out and raising this issue. For the reasons @nateprewitt mentioned, and because raising the floor of a version pin can be breaking, we've chosen to leave the pins where they are.

[github-actions]

This issue is now closed. Comments on closed issues are hard for our team to see.
If you need more assistance, please open a new issue that references this one.