Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

ci: Use uv for all pip installs #2444

Merged
merged 20 commits into from
Apr 30, 2024
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
20 commits
Select commit Hold shift + click to select a range
fea7834
Add uv to dev dependencies
matthewfeickert Feb 16, 2024
02a891a
Ad uv to all pip install commands
matthewfeickert Feb 16, 2024
8eee37a
install quiet too
matthewfeickert Feb 16, 2024
66811f2
Add uv to pip uninstall
matthewfeickert Feb 16, 2024
3ed06bd
install uv
matthewfeickert Feb 16, 2024
96b6f0d
Require v0.1.12 for --system
matthewfeickert Feb 28, 2024
4ce9ccd
Specify 'install --system' to use CI provided Python
matthewfeickert Feb 28, 2024
7117d0c
Use '--no-cache' as '--no-cache-dir' is for normal pip only
matthewfeickert Feb 28, 2024
d803c48
Use 'pyhf[extras] @ .' pattern over '.[extras]' pattern
matthewfeickert Feb 28, 2024
3758cd4
Revert "Use 'pyhf[extras] @ .' pattern over '.[extras]' pattern"
matthewfeickert Mar 28, 2024
249241e
Update uv lower bound
matthewfeickert Mar 28, 2024
9b7570a
Use 'installer uv' option for build
matthewfeickert Mar 29, 2024
fd7b114
quote local installs with extras
matthewfeickert Apr 2, 2024
3bd1ff1
Update uv
matthewfeickert Apr 27, 2024
3eef2a7
fix: Fall back to pip for Python 3.8 as uv fails here
matthewfeickert Apr 27, 2024
b8f2958
Just give in and use 'uv pip' for everything
matthewfeickert Apr 27, 2024
d4f6e7c
fix: Still need to call build from 'python -m' if not using pipx
matthewfeickert Apr 27, 2024
aa7955d
fix: uv pip uninstall needs 'system' but not 'yes'
matthewfeickert Apr 27, 2024
49b8ed2
ci: Adapt to uv's --upgrade rules to allow scipy without new numpy
matthewfeickert Apr 27, 2024
854906d
Flip index-url and extra-index-url as uv and pip differ
matthewfeickert Apr 27, 2024
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
5 changes: 3 additions & 2 deletions .github/workflows/bump-version.yml
Original file line number Diff line number Diff line change
Expand Up @@ -204,8 +204,9 @@ jobs:

- name: Install Python dependencies
run: |
python -m pip install --upgrade pip setuptools wheel
python -m pip install tbump
python -m pip install uv
uv pip install --system --upgrade pip setuptools wheel
uv pip install --system tbump
python -m pip list

- name: Setup Git user to push new tag
Expand Down
5 changes: 3 additions & 2 deletions .github/workflows/ci-windows.yml
Original file line number Diff line number Diff line change
Expand Up @@ -29,8 +29,9 @@ jobs:

- name: Install dependencies
run: |
python -m pip install --upgrade pip setuptools wheel
python -m pip install --upgrade '.[all,test]'
python -m pip install uv
uv pip install --system --upgrade pip setuptools wheel
uv pip install --system --upgrade '.[all,test]'

- name: List installed Python packages
run: python -m pip list
Expand Down
9 changes: 9 additions & 0 deletions .github/workflows/ci.yml
Original file line number Diff line number Diff line change
Expand Up @@ -44,6 +44,15 @@ jobs:
python-version: ${{ matrix.python-version }}

- name: Install dependencies
if: matrix.python-version != '3.8'
run: |
python -m pip install uv
uv pip install --system --upgrade pip setuptools wheel
uv pip install --system --upgrade ".[all,test]"

# c.f. https://github.com/astral-sh/uv/issues/2062
- name: Install dependencies (Python 3.8)
if: matrix.python-version == '3.8'
Comment on lines +47 to +55
Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

A bit of a hack, but until astral-sh/uv#2062 gets resolves, we might as well speed up everything else.

run: |
python -m pip install --upgrade pip setuptools wheel
python -m pip install --upgrade ".[all,test]"
Expand Down
64 changes: 35 additions & 29 deletions .github/workflows/dependencies-head.yml
Original file line number Diff line number Diff line change
Expand Up @@ -29,8 +29,9 @@ jobs:

- name: Install dependencies
run: |
python -m pip install --upgrade pip setuptools wheel
python -m pip --no-cache-dir --quiet install --upgrade --pre ".[all,test]"
python -m pip install uv
uv pip install --system --upgrade pip setuptools wheel
uv pip --no-cache --quiet install --system --upgrade --pre ".[all,test]"
python -m pip list

- name: List release candidates, alpha, and beta releases
Expand All @@ -57,14 +58,14 @@ jobs:
with:
python-version: ${{ matrix.python-version }}

# Use nightly SciPy wheels from Anaconda's PyPI
# c.f. https://twitter.com/ralfgommers/status/1419917265781334025
- name: Install dependencies
run: |
python -m pip install --upgrade pip setuptools wheel
python -m pip --no-cache-dir --quiet install --upgrade ".[all,test]"
python -m pip uninstall --yes scipy
python -m pip install --upgrade --index-url https://pypi.anaconda.org/scientific-python-nightly-wheels/simple scipy
python -m pip install uv
uv pip install --system --upgrade pip setuptools wheel
uv pip --no-cache --quiet install --system --upgrade ".[all,test]"
uv pip uninstall --system scipy
# uv wants to upgrade dependencies (numpy) to a dev release too, so don't --upgrade
uv pip install --system --index-url https://pypi.anaconda.org/scientific-python-nightly-wheels/simple scipy
Comment on lines +67 to +68
Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This was previously forcing an upgrade of numpy with uv as well, but just want to test scipy which should work with numpy v1 or v2, and trying to test numpy v2 will break jax at the moment.

python -m pip list

- name: Test with pytest
Expand All @@ -87,11 +88,12 @@ jobs:
python-version: ${{ matrix.python-version }}
- name: Install dependencies
run: |
python -m pip install --upgrade pip setuptools wheel
python -m pip --no-cache-dir --quiet install --upgrade ".[all,test]"
python -m pip uninstall --yes iminuit
python -m pip install --upgrade cython
python -m pip install --upgrade git+https://github.com/scikit-hep/iminuit.git
python -m pip install uv
uv pip install --system --upgrade pip setuptools wheel
uv pip --no-cache --quiet install --system --upgrade ".[all,test]"
uv pip uninstall --system iminuit
uv pip install --system --upgrade cython
uv pip install --system --upgrade git+https://github.com/scikit-hep/iminuit.git
python -m pip list
- name: Test with pytest
run: |
Expand All @@ -113,10 +115,11 @@ jobs:
python-version: ${{ matrix.python-version }}
- name: Install dependencies
run: |
python -m pip install --upgrade pip setuptools wheel
python -m pip --no-cache-dir --quiet install --upgrade ".[all,test]"
python -m pip uninstall --yes uproot
python -m pip install --upgrade git+https://github.com/scikit-hep/uproot5.git
python -m pip install uv
uv pip install --system --upgrade pip setuptools wheel
uv pip --no-cache --quiet install --system --upgrade ".[all,test]"
uv pip uninstall --system uproot
uv pip install --system --upgrade git+https://github.com/scikit-hep/uproot5.git
python -m pip list
- name: Test with pytest
run: |
Expand All @@ -140,16 +143,18 @@ jobs:

- name: Install dependencies
run: |
python -m pip install --upgrade pip setuptools wheel
python -m pip --no-cache-dir --quiet install --upgrade ".[all,test]"
python -m pip uninstall --yes matplotlib
# Need to use --extra-index-url as dependencies aren't on scientific-python-nightly-wheels package index.
python -m pip install uv
uv pip install --system --upgrade pip setuptools wheel
uv pip --no-cache --quiet install --system --upgrade ".[all,test]"
uv pip uninstall --system matplotlib
# Need to use --extra-index-url as all dependencies aren't on scientific-python-nightly-wheels package index.
# Need to use --pre as dev releases will need priority over stable releases.
python -m pip install \
--upgrade \
# Note that uv and pip differ on --extra-index-url priority
# c.f. https://github.com/scientific-python/upload-nightly-action/issues/76
uv pip install --system \
Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Double check this as pip and uv handle --extra-index-url differently. c.f. scientific-python/upload-nightly-action#76

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Yeah, this needed to get flipped to

        uv pip install --system \
          --pre \
          --index-url https://pypi.org/simple/ \
          --extra-index-url https://pypi.anaconda.org/scientific-python-nightly-wheels/simple \
          matplotlib

as uv give priority to --extra-index-url.

$ pip list
Package         Version
--------------- -------------------------
contourpy       1.3.0.dev1
cycler          0.12.1
fonttools       4.51.0
kiwisolver      1.4.5
matplotlib      3.10.0.dev114+g4cbef2d4c0
numpy           2.1.0.dev0
packaging       24.0
pillow          10.3.0
pip             24.0
pyparsing       3.1.2
python-dateutil 2.9.0.post0
setuptools      69.5.1
six             1.16.0
uv              0.1.39
wheel           0.43.0

--pre \
--index-url https://pypi.anaconda.org/scientific-python-nightly-wheels/simple \
--extra-index-url https://pypi.org/simple/ \
--index-url https://pypi.org/simple/ \
--extra-index-url https://pypi.anaconda.org/scientific-python-nightly-wheels/simple \
matplotlib

- name: List installed Python packages
Expand All @@ -175,10 +180,11 @@ jobs:
python-version: ${{ matrix.python-version }}
- name: Install dependencies
run: |
python -m pip install --upgrade pip setuptools wheel
python -m pip --no-cache-dir --quiet install --upgrade ".[all,test]"
python -m pip uninstall --yes pytest
python -m pip install --upgrade git+https://github.com/pytest-dev/pytest.git
python -m pip install uv
uv pip install --system --upgrade pip setuptools wheel
uv pip --no-cache --quiet install --system --upgrade ".[all,test]"
uv pip uninstall --system pytest
uv pip install --system --upgrade git+https://github.com/pytest-dev/pytest.git
python -m pip list
- name: Test with pytest
run: |
Expand Down
7 changes: 4 additions & 3 deletions .github/workflows/docs.yml
Original file line number Diff line number Diff line change
Expand Up @@ -28,9 +28,10 @@ jobs:

- name: Install Python dependencies
run: |
python -m pip install --upgrade pip setuptools wheel
python -m pip --quiet install --upgrade .[docs,test]
python -m pip install yq
python -m pip install uv
uv pip install --system --upgrade pip setuptools wheel
uv pip --quiet install --system --upgrade ".[docs,test]"
uv pip install --system yq
python -m pip list

- name: Install apt-get dependencies
Expand Down
5 changes: 3 additions & 2 deletions .github/workflows/lower-bound-requirements.yml
Original file line number Diff line number Diff line change
Expand Up @@ -26,8 +26,9 @@ jobs:

- name: Install dependencies and force lowest bound
run: |
python -m pip install --upgrade pip setuptools wheel
python -m pip --no-cache-dir install --constraint tests/constraints.txt ".[all,test]"
python -m pip install uv
uv pip install --system --upgrade pip setuptools wheel
uv pip --no-cache install --system --constraint tests/constraints.txt ".[all,test]"

- name: List installed Python packages
run: python -m pip list
Expand Down
5 changes: 3 additions & 2 deletions .github/workflows/notebooks.yml
Original file line number Diff line number Diff line change
Expand Up @@ -28,9 +28,10 @@ jobs:

- name: Install dependencies
run: |
python -m pip install --upgrade pip setuptools wheel
python -m pip install uv
uv pip install --system --upgrade pip setuptools wheel
# FIXME: c.f. https://github.com/scikit-hep/pyhf/issues/2104
python -m pip install --upgrade ".[all,test]" 'jupyter-client<8.0.0'
uv pip install --system --upgrade ".[all,test]" 'jupyter-client<8.0.0'

- name: List installed Python packages
run: python -m pip list
Expand Down
9 changes: 5 additions & 4 deletions .github/workflows/publish-package.yml
Original file line number Diff line number Diff line change
Expand Up @@ -43,19 +43,20 @@ jobs:

- name: Install python-build and twine
run: |
python -m pip install --upgrade pip
python -m pip install build twine
python -m pip install uv
uv pip install --system --upgrade pip
uv pip install --system build twine
python -m pip list

- name: Build a sdist and wheel
if: github.event_name != 'schedule'
run: |
python -m build .
python -m build --installer uv .

- name: Build a sdist and wheel and check for warnings
if: github.event_name == 'schedule'
run: |
PYTHONWARNINGS=error,default::DeprecationWarning python -m build .
PYTHONWARNINGS=error,default::DeprecationWarning python -m build --installer uv .

- name: Verify untagged commits have dev versions
if: "!startsWith(github.ref, 'refs/tags/')"
Expand Down
9 changes: 5 additions & 4 deletions .github/workflows/release_tests.yml
Original file line number Diff line number Diff line change
Expand Up @@ -36,9 +36,10 @@ jobs:

- name: Install from PyPI
run: |
python -m pip install --upgrade pip setuptools wheel
python -m pip install --pre 'pyhf[backends,xmlio]'
python -m pip install pytest
python -m pip install uv
uv pip install --system --upgrade pip setuptools wheel
uv pip install --system --pre 'pyhf[backends,xmlio]'
uv pip install --system pytest
python -m pip list

- name: Canary test public API
Expand All @@ -48,6 +49,6 @@ jobs:
# FIXME: c.f. https://github.com/proycon/codemetapy/issues/24
- name: Verify requirements in codemeta.json
run: |
python -m pip install jq "codemetapy>=2.3.0"
uv pip install --system jq "codemetapy>=2.3.0"
codemetapy --inputtype python --no-extras pyhf > codemeta_generated.json
diff <(jq -S .softwareRequirements codemeta.json) <(jq -S .softwareRequirements codemeta_generated.json)
2 changes: 1 addition & 1 deletion .readthedocs.yaml
Original file line number Diff line number Diff line change
Expand Up @@ -21,7 +21,7 @@ sphinx:
# If using Sphinx, optionally build your docs in additional formats such as PDF and ePub
formats: all

# python -m pip install .[docs]
# python -m pip install '.[docs]'
python:
install:
- method: pip
Expand Down
1 change: 1 addition & 0 deletions pyproject.toml
Original file line number Diff line number Diff line change
Expand Up @@ -134,6 +134,7 @@ develop = [
"pre-commit",
"nox",
"codemetapy>=2.3.0",
"uv>=0.1.39"
]

[tool.hatch.version]
Expand Down