Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

chore: bump Python version to 3.12 and update requirements #2770

Merged
merged 52 commits into from
Dec 13, 2023

Conversation

AJPfleger
Copy link
Contributor

@AJPfleger AJPfleger commented Dec 5, 2023

This PR updates the existing requirements for python-based CI jobs to the newest possible version. It also tries to enforce python 3.12 (newest) wherever possible.

Performance boost

linux_examples_tests: 18.5 min -> 15 min

Important changes/notes

The pytest-check 2.2.0..2.2.2 soft-break some CI checks. Therefore, we fix it to 2.1.5. More to this in issue #2777 :

  • linux_test_examples (verified)
  • test_exatrkx_python (very likely)

uproot 5.x and awkward 2.x change the root-hashes.

The test test_exatrkx_python got a new separate requirements-file, because the test only runs on ubuntu 20.04 which uses python 3.8. The newest numpy (for awkward and uproot) doesn't support python 3.8 anymore. When we migrate this test to a newer OS, we can use the shared requirements.txt again.

clang_tidy had to be adapted at a few points, since some ofpydantic's functionalities were deprecated.

Side note to pip-compile

Before generating a new requirements.txt one should delete the old one. Otherwise, the versions for the packages are not updated (makes sense to some extent).

Blocked by other PRs

We created files, that were referred to at places we did not check.

Other Questions

The release-workflows still use 3.8. Should we upgrade as well, or are we too afraid, to break something in there?
https://github.com/acts-project/acts/blob/main/.github/workflows/release.yml
https://github.com/acts-project/acts/blob/main/.github/workflows/release_pr.yml
-> An update to the release script will come in

@github-actions github-actions bot added Infrastructure Changes to build tools, continous integration, ... Component - Examples Affects the Examples module Component - Documentation Affects the documentation labels Dec 5, 2023
Copy link

codecov bot commented Dec 5, 2023

Codecov Report

All modified and coverable lines are covered by tests ✅

Comparison is base (ed4761a) 48.62% compared to head (98d783e) 48.62%.

Additional details and impacted files
@@           Coverage Diff           @@
##             main    #2770   +/-   ##
=======================================
  Coverage   48.62%   48.62%           
=======================================
  Files         479      479           
  Lines       27942    27942           
  Branches    13213    13213           
=======================================
  Hits        13588    13588           
  Misses       4793     4793           
  Partials     9561     9561           

☔ View full report in Codecov by Sentry.
📢 Have feedback on the report? Share it here.

@paulgessinger paulgessinger added this to the next milestone Dec 6, 2023
@AJPfleger AJPfleger marked this pull request as draft December 7, 2023 08:54
@AJPfleger AJPfleger marked this pull request as ready for review December 7, 2023 10:46
@AJPfleger AJPfleger changed the title [WIP] chore: bump Python version to 3.12 and update requirements chore: bump Python version to 3.12 and update requirements Dec 7, 2023
Copy link
Contributor

@andiwand andiwand left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

no objections from my side

@kodiakhq kodiakhq bot merged commit 636eff4 into acts-project:main Dec 13, 2023
50 checks passed
@acts-project-service
Copy link
Collaborator

acts-project-service commented Dec 13, 2023

🔴 Athena integration test results

🔴 Some tests have failed!

Please investigate the pipeline!

status job report
🟢 run_unit_tests
🟢 run_ci_tests: ../athena/AtlasTest/CITest/test/ActsBenchmarkWithSpot.sh 8 100
🟢 run_ci_tests: ../athena/AtlasTest/CITest/test/ActsWorkflow.sh
🟢 run_ci_tests: ../athena/AtlasTest/CITest/test/ActsValidateAmbiguityResolution.sh
🟢 run_ci_tests: ../athena/AtlasTest/CITest/test/ActsValidateResolvedTracks.sh
🟢 run_ci_tests: ../athena/AtlasTest/CITest/test/ActsValidateTracks.sh
🟢 run_ci_tests: ../athena/AtlasTest/CITest/test/ActsValidateActsCoreSpacePoints.sh
🟢 run_ci_tests: ../athena/AtlasTest/CITest/test/ActsValidateActsSpacePoints.sh
🟢 run_ci_tests: ../athena/AtlasTest/CITest/test/ActsValidateSeeds.sh
🟢 run_ci_tests: ../athena/AtlasTest/CITest/test/ActsValidateOrthogonalSeeds.sh
🟢 run_ci_tests: ../athena/AtlasTest/CITest/test/ActsValidateClusters.sh
🟢 run_ci_tests: ../athena/AtlasTest/CITest/test/ActsPersistifyEDM.sh
🟢 run_ci_tests: ../athena/AtlasTest/CITest/test/ActsGSFRefitting.sh
🟢 run_ci_tests: ../athena/AtlasTest/CITest/test/ActsKfRefitting.sh
🟢 run_ci_tests: python3 ../athena/Tracking/Acts/ActsGeometry/test/ActsExtrapolationAlgTest.py
🟢 run_ci_tests: python3 ../athena/Tracking/Acts/ActsGeometry/test/ActsITkTest.py
🟢 run_workflow_tests_run4_mc
🟢 run_workflow_tests_run2_mc
🟢 run_workflow_tests_run2_data
🔴 run_workflow_tests_run3_mc
🟢 run_workflow_tests_run3_data
🟢 run_art_test: test_data18_13TeV_1000evt
🟢 run_art_test: test_ttbarPU40_reco

@acts-project-service acts-project-service added the Fails Athena tests This PR causes a failure in the Athena tests label Dec 13, 2023
@AJPfleger AJPfleger deleted the py-version branch December 19, 2023 14:36
@paulgessinger paulgessinger modified the milestones: next, v32.0.0 Jan 19, 2024
@paulgessinger paulgessinger removed the Fails Athena tests This PR causes a failure in the Athena tests label Jan 23, 2024
LaraCalic pushed a commit to LaraCalic/acts that referenced this pull request Feb 10, 2024
…ect#2770)

This PR updates the existing requirements for python-based CI jobs to the newest possible version. It also tries to enforce `python 3.12` (newest) wherever possible.

## Performance boost

linux_examples_tests: 18.5 min -> 15 min

## Important changes/notes

The `pytest-check 2.2.0..2.2.2` soft-break some CI checks. Therefore, we fix it to `2.1.5.` More to this in issue acts-project#2777 :
- linux_test_examples (verified)
- test_exatrkx_python (very likely)

`uproot 5.x` and `awkward 2.x` change the root-hashes.

The test `test_exatrkx_python` got a new separate requirements-file, because the test only runs on `ubuntu 20.04` which uses `python 3.8`. The newest `numpy` (for `awkward` and `uproot`) doesn't support `python 3.8` anymore. When we migrate this test to a newer `OS`, we can use the shared `requirements.txt` again.

`clang_tidy` had to be adapted at a few points, since some of`pydantic`'s functionalities were deprecated.


## Side note to `pip-compile`

Before generating a new `requirements.txt` one should delete the old one. Otherwise, the versions for the packages are not updated (makes sense to some extent).

## Blocked by other PRs
We created files, that were referred to at places we did not check.
- acts-project#2778

## Other Questions

The release-workflows still use `3.8`. Should we upgrade as well, or are we too afraid, to break something in there?
https://github.com/acts-project/acts/blob/main/.github/workflows/release.yml
https://github.com/acts-project/acts/blob/main/.github/workflows/release_pr.yml
-> An update to the release script will come in
- acts-project#2779
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
Changes Performance Component - Documentation Affects the documentation Component - Examples Affects the Examples module Infrastructure Changes to build tools, continous integration, ...
Projects
None yet
Development

Successfully merging this pull request may close these issues.

4 participants