CI strategy

This page is a work in progress — 2024-08-25

---

We use continuous integration to continuously run a range of tests and checks on our code.

Overview

The standard .github/workflows/gh-ci.yaml workflow configuration file for GitHub-actions and azure-pipelines.yml for Azure contain all information about our CI runners.

We are also using CirrusCI (config files .cirrus.star, maintainer/ci/cirrus-ci.yml, maintainer/ci/cirrus-deploy.yml) for some special cases.

Types of tests/checks

1. functional/unit tests (pytest)
2. coverage (of the unit tests)
3. doctests
4. performance benchmarks (asv)
5. sdist package build

Priorities

Currently we first make sure that MDAnalysis runs correctly with all supported Python versions. We always test all versions on Linux x86_64.

We then want to make sure that it runs on macOS (M1/M2 macs, older x86_64) and Windows (x86_64). In order to decrease the load on CI, we do not test the full matrix of Python versions with operating systems and CPU architectures.

External variables: Test matrix

Elements of the test matrix are the operating system, architecture, and the source of the dependency installation. We do not test a full Cartesian product of all possibilities. Instead different runners cover subsets.

(TODO: describe which runner does what and why)

Operating systems

• Linux
• macOS
• Windows

Python version

Follow SPEC0, so typically only the last 3-4 minor versions, e.g., 3.10, 3.11, 3.12.

Architectures

• x86_64
• arm64 (macOS) – GH runners
• arm64 (Linux) — CirrusCI

numpy version

• minimal
• latest

Installation source of dependencies

• PyPi (pip installation)
• conda-forge (mamba installation)

Implementation

? Which resources/runners cover the priorities ?