From faf1150209c85f26e4ccdf45e6398b2e102a5480 Mon Sep 17 00:00:00 2001 From: "Alicia A. Evans" <108547992+aliciaaevans@users.noreply.github.com> Date: Wed, 10 Jul 2024 14:21:59 -0400 Subject: [PATCH] OSX-Arm64 updates (#24) * Add arm-osx64 to docs * update ci docs for osx-arm64 * update title * fix formatting * format to avoid wrapping * remove extra backtick --- source/developer/aarch64.rst | 17 ++++++++++------- source/developer/bulk.rst | 2 +- source/developer/ci-inventory.rst | 24 ++++++++++++++---------- source/developer/repodata_patching.rst | 2 +- source/faqs.rst | 3 +-- source/index.rst | 2 +- 6 files changed, 28 insertions(+), 22 deletions(-) diff --git a/source/developer/aarch64.rst b/source/developer/aarch64.rst index 61bd6e5..bbbd86d 100644 --- a/source/developer/aarch64.rst +++ b/source/developer/aarch64.rst @@ -1,17 +1,19 @@ -``aarch64`` builds +``aarch64`` and ``arm64`` builds ================== .. datechanged:: 2024-03-04 Added this section as initial linux-aarch64 builds are starting -While we do not yet have builds for ``osx-arm64`` (see -:ref:`platform-nomenclature-faq`), we are starting to roll out -``linux-aarch64`` builds. These run on CircleCI, which offers the -infrastructure for this. +.. datechanged:: 2024-04-04 + Information about osx-arm64 builds + +We are starting to roll out ``linux-aarch64`` and ``osx-arm64`` (see +:ref:`platform-nomenclature-faq`) on CircleCI and GitHub Actions, respectively. +These additional CI platforms offer the runner types needed for these architectures. This is being initially approached as an opt-in process as we make sure all the moving parts are working correctly. A recipe can be flagged for -building on ``linux-aarch64`` by adding the following to the +building on ``linux-aarch64`` and/or ``osx-arm64`` by adding the following to the :file:`meta.yaml` file: .. code-block:: yaml @@ -19,10 +21,11 @@ building on ``linux-aarch64`` by adding the following to the extra: additional-platforms: - linux-aarch64 + - osx-arm64 The current CircleCI config will only run if at least one recipe in the commit range (typically for the PR) includes the above additional platform. Support for building and uploading ``linux-aarch64`` containers is planned but -not yet implemented (as of March 2024). +not yet implemented (as of July 2024). diff --git a/source/developer/bulk.rst b/source/developer/bulk.rst index d25f2ad..c4feeb5 100644 --- a/source/developer/bulk.rst +++ b/source/developer/bulk.rst @@ -232,7 +232,7 @@ Some unordered notes on working with the bulk branch: that don't need to be rebuilt, but that work needs to be done simply to figure out if a rebuild is needed, and so this is expected. -- For ``linux-64`` and ``osx-64``, the bulk runs take place on GitHub Actions, +- For ``linux-64``, ``osx-64``, and ``osx-arm64`` the bulk runs take place on GitHub Actions, and the configuration is in :file:`.github/workflows/Bulk.yml`. For ``linux-aarch64``, the builds take place on CircleCI and the configuration is in :file:`.circleci/config.yml`. diff --git a/source/developer/ci-inventory.rst b/source/developer/ci-inventory.rst index e6101d6..cb6ff77 100644 --- a/source/developer/ci-inventory.rst +++ b/source/developer/ci-inventory.rst @@ -12,6 +12,9 @@ CI Inventory .. datechanged:: 2024-03-04 Start using ``linux-aarch64`` in bulk and recipe tests +.. datechanged:: 2024-07-05 + Start using ``osx-arm64`` in bulk and recipe tests + This page documents the various moving parts that, together, make Bioconda work. We rely on a mixture of free services to spread the workload and to maintain flexibility over the long term in case a service becomes unusable. @@ -70,8 +73,8 @@ maintain flexibility over the long term in case a service becomes unusable. - ``bioconda-recipes`` - on push - `azure-pipeline.yml `_ (``linux-64``, ``osx-64``); - `config.yml `_ (``linux-aarch64``) - - ``linux-64``, ``osx-64``, ``linux-aarch64`` + `config.yml `_ (``linux-aarch64``, ``osx-arm64``) + - ``linux-64``, ``osx-64``, ``linux-aarch64``, ``osx-arm64`` - These are the most-run tests: these are what run on every change on pull requests to bioconda-recipes, and they must pass before the recipe is merged into the master branch. @@ -90,14 +93,15 @@ maintain flexibility over the long term in case a service becomes unusable. * - Master branch tests - - Azure Pipelines + - Azure Pipelines, CircleCI - ``bioconda-recipes`` - push to master (bioconda-recipes) - - `azure-pipeline-master.yml `_ - - ``linux-64``, ``osx-64``, ``linux-aarch64`` - - Runs the same tests as for PRs, but on the master branch. Note that - when the bot is doing the merge, it adds ``[ci skip]`` in the commit - comment, and works with the already-built artifacts from the PR. + - `azure-pipeline-master.yml `_ (``linux-64``, ``osx-64``); + `config.yml `_ (``linux-aarch64``, ``osx-arm64``) + - ``linux-64``, ``osx-64``, ``linux-aarch64``, ``osx-arm64`` + - Runs when a PR is merged to the master branch. The already-built artifacts are retrieved from the PR and uploaded to the ``bioconda`` channel. + Container(s) are uploaded to quay.io/biocontainers. + If artifacts are not found, it will build the recipe(s) as a fallback. * - Mergify @@ -210,9 +214,9 @@ maintain flexibility over the long term in case a service becomes unusable. - GitHub Actions - ``bioconda-recipes`` - push to ``bulk`` branch of bioconda-utils - - `Bulk.yml `_ (``linux-64``, ``osx-64``); + - `Bulk.yml `_ (``linux-64``, ``osx-64``, ``osx-arm64``); `config.yml `_ (``linux-aarch64``) - - ``linux-64``, ``osx-64``, ``linux-aarch64`` + - ``linux-64``, ``osx-64``, ``linux-aarch64``, ``osx-arm64`` - If pushing to the special ``bulk`` branch, this workflow will run. It uses special bioconda-utils functionality to split the full DAG into sub-DAGs and submits them to independent parallel jobs. When recipes diff --git a/source/developer/repodata_patching.rst b/source/developer/repodata_patching.rst index 539a05a..5944076 100644 --- a/source/developer/repodata_patching.rst +++ b/source/developer/repodata_patching.rst @@ -21,7 +21,7 @@ What is repodata? ----------------- Repodata is a JSON file that contains a variety of information for each package -in Bioconda. There is one for each architecture (linux-64, linux-aarch64, osx-64, noarch, etc.) +in Bioconda. There is one for each architecture (``linux-64``, ``linux-aarch64``, ``osx-64``, ``osx-arm64``, ``noarch``, etc.) and they're hosted in the bioconda channel. For example, the noarch repodata.json file `is available here `_. Let's take a look at the what sorts of things are stored within this file for a diff --git a/source/faqs.rst b/source/faqs.rst index 06561b4..e5ba0a5 100644 --- a/source/faqs.rst +++ b/source/faqs.rst @@ -82,8 +82,7 @@ What versions are supported? **Operating Systems:** Bioconda supports Linux (x86_64 and aarch64/arm64) and -macOS (x86_64). ARM is not currently supported for macOS. Windows is not -supported. +macOS (x86_64 and arm64). Windows is not supported. **Python:** Bioconda currently supports Python 3.8, 3.9, and 3.10 (see "pinned packages" diff --git a/source/index.rst b/source/index.rst index 991416a..bb6d4ac 100644 --- a/source/index.rst +++ b/source/index.rst @@ -20,7 +20,7 @@ **Bioconda** lets you install thousands of software packages related to biomedical research using the `conda `_ package manager. -**NOTE**: *Bioconda supports only Linux (64-bit and AArch64) and macOS (x86_64)* +**NOTE**: *Bioconda supports only Linux (64-bit and AArch64) and macOS (x86_64 and ARM64)* Usage =====