From 38428061e0ee485fcd6a087260d45de37d5b3ca5 Mon Sep 17 00:00:00 2001 From: aliciaaevans Date: Fri, 5 Jul 2024 18:09:13 -0400 Subject: [PATCH 1/6] Add arm-osx64 to docs --- source/developer/aarch64.rst | 15 +++++++++------ source/developer/bulk.rst | 2 +- source/developer/repodata_patching.rst | 2 +- source/faqs.rst | 3 +-- source/index.rst | 2 +- 5 files changed, 13 insertions(+), 11 deletions(-) diff --git a/source/developer/aarch64.rst b/source/developer/aarch64.rst index 61bd6e5..220d348 100644 --- a/source/developer/aarch64.rst +++ b/source/developer/aarch64.rst @@ -4,14 +4,16 @@ .. datechanged:: 2024-03-04 Added this section as initial linux-aarch64 builds are starting -While we do not yet have builds for ``osx-arm64`` (see -:ref:`platform-nomenclature-faq`), we are starting to roll out -``linux-aarch64`` builds. These run on CircleCI, which offers the -infrastructure for this. +.. datechanged:: 2024-04-04 + Information about osx-arm64 builds + +We are starting to roll out ``linux-aarch64`` and ``osx-arm64`` (see +:ref:`platform-nomenclature-faq`) on CircleCI and GitHub Actions, respectively. +These additional CI platforms offer the runner types needed for these architectures. This is being initially approached as an opt-in process as we make sure all the moving parts are working correctly. A recipe can be flagged for -building on ``linux-aarch64`` by adding the following to the +building on ``linux-aarch64`` and/or ``osx-arm64`` by adding the following to the :file:`meta.yaml` file: .. code-block:: yaml @@ -19,10 +21,11 @@ building on ``linux-aarch64`` by adding the following to the extra: additional-platforms: - linux-aarch64 + - osx-arm64 The current CircleCI config will only run if at least one recipe in the commit range (typically for the PR) includes the above additional platform. Support for building and uploading ``linux-aarch64`` containers is planned but -not yet implemented (as of March 2024). +not yet implemented (as of July 2024). diff --git a/source/developer/bulk.rst b/source/developer/bulk.rst index d25f2ad..c4feeb5 100644 --- a/source/developer/bulk.rst +++ b/source/developer/bulk.rst @@ -232,7 +232,7 @@ Some unordered notes on working with the bulk branch: that don't need to be rebuilt, but that work needs to be done simply to figure out if a rebuild is needed, and so this is expected. -- For ``linux-64`` and ``osx-64``, the bulk runs take place on GitHub Actions, +- For ``linux-64``, ``osx-64``, and ``osx-arm64`` the bulk runs take place on GitHub Actions, and the configuration is in :file:`.github/workflows/Bulk.yml`. For ``linux-aarch64``, the builds take place on CircleCI and the configuration is in :file:`.circleci/config.yml`. diff --git a/source/developer/repodata_patching.rst b/source/developer/repodata_patching.rst index 539a05a..7851fa8 100644 --- a/source/developer/repodata_patching.rst +++ b/source/developer/repodata_patching.rst @@ -21,7 +21,7 @@ What is repodata? ----------------- Repodata is a JSON file that contains a variety of information for each package -in Bioconda. There is one for each architecture (linux-64, linux-aarch64, osx-64, noarch, etc.) +in Bioconda. There is one for each architecture (linux-64, linux-aarch64, osx-64, osx-arm64, noarch, etc.) and they're hosted in the bioconda channel. For example, the noarch repodata.json file `is available here `_. Let's take a look at the what sorts of things are stored within this file for a diff --git a/source/faqs.rst b/source/faqs.rst index 06561b4..e5ba0a5 100644 --- a/source/faqs.rst +++ b/source/faqs.rst @@ -82,8 +82,7 @@ What versions are supported? **Operating Systems:** Bioconda supports Linux (x86_64 and aarch64/arm64) and -macOS (x86_64). ARM is not currently supported for macOS. Windows is not -supported. +macOS (x86_64 and arm64). Windows is not supported. **Python:** Bioconda currently supports Python 3.8, 3.9, and 3.10 (see "pinned packages" diff --git a/source/index.rst b/source/index.rst index 991416a..bb6d4ac 100644 --- a/source/index.rst +++ b/source/index.rst @@ -20,7 +20,7 @@ **Bioconda** lets you install thousands of software packages related to biomedical research using the `conda `_ package manager. -**NOTE**: *Bioconda supports only Linux (64-bit and AArch64) and macOS (x86_64)* +**NOTE**: *Bioconda supports only Linux (64-bit and AArch64) and macOS (x86_64 and ARM64)* Usage ===== From 21d20a41b4462351413bc0b1c38a410bb554b5c1 Mon Sep 17 00:00:00 2001 From: aliciaaevans Date: Mon, 8 Jul 2024 11:52:21 -0400 Subject: [PATCH 2/6] update ci docs for osx-arm64 --- source/developer/ci-inventory.rst | 24 ++++++++++++++---------- 1 file changed, 14 insertions(+), 10 deletions(-) diff --git a/source/developer/ci-inventory.rst b/source/developer/ci-inventory.rst index e6101d6..d223e8b 100644 --- a/source/developer/ci-inventory.rst +++ b/source/developer/ci-inventory.rst @@ -12,6 +12,9 @@ CI Inventory .. datechanged:: 2024-03-04 Start using ``linux-aarch64`` in bulk and recipe tests +.. datechanged:: 2024-07-05 + Start using ``osx-arm64`` in bulk and recipe tests + This page documents the various moving parts that, together, make Bioconda work. We rely on a mixture of free services to spread the workload and to maintain flexibility over the long term in case a service becomes unusable. @@ -70,8 +73,8 @@ maintain flexibility over the long term in case a service becomes unusable. - ``bioconda-recipes`` - on push - `azure-pipeline.yml `_ (``linux-64``, ``osx-64``); - `config.yml `_ (``linux-aarch64``) - - ``linux-64``, ``osx-64``, ``linux-aarch64`` + `config.yml `_ (``linux-aarch64``, ``osx-arm64``) + - ``linux-64``, ``osx-64``, ``linux-aarch64``, ``osx-arm64`` - These are the most-run tests: these are what run on every change on pull requests to bioconda-recipes, and they must pass before the recipe is merged into the master branch. @@ -90,14 +93,15 @@ maintain flexibility over the long term in case a service becomes unusable. * - Master branch tests - - Azure Pipelines + - Azure Pipelines, CircleCI - ``bioconda-recipes`` - push to master (bioconda-recipes) - - `azure-pipeline-master.yml `_ - - ``linux-64``, ``osx-64``, ``linux-aarch64`` - - Runs the same tests as for PRs, but on the master branch. Note that - when the bot is doing the merge, it adds ``[ci skip]`` in the commit - comment, and works with the already-built artifacts from the PR. + - `azure-pipeline-master.yml `; + `config.yml `_ (``linux-aarch64``, ``osx-arm64``) + - ``linux-64``, ``osx-64``, ``linux-aarch64``, ``osx-arm64` + - Runs when a PR is merged to the master branch. The already-built artifacts are retrieved from the PR and uploaded to the ``bioconda`` channel. + Container(s) are uploaded to quay.io/biocontainers. + If artifacts are not found, it will build the recipe(s) as a fallback. * - Mergify @@ -210,9 +214,9 @@ maintain flexibility over the long term in case a service becomes unusable. - GitHub Actions - ``bioconda-recipes`` - push to ``bulk`` branch of bioconda-utils - - `Bulk.yml `_ (``linux-64``, ``osx-64``); + - `Bulk.yml `_ (``linux-64``, ``osx-64``, ``osx-arm64``); `config.yml `_ (``linux-aarch64``) - - ``linux-64``, ``osx-64``, ``linux-aarch64`` + - ``linux-64``, ``osx-64``, ``linux-aarch64``, ``osx-arm64`` - If pushing to the special ``bulk`` branch, this workflow will run. It uses special bioconda-utils functionality to split the full DAG into sub-DAGs and submits them to independent parallel jobs. When recipes From 0a1478f2524ce316a0caa9841b1ec2ff155b8f80 Mon Sep 17 00:00:00 2001 From: aliciaaevans Date: Mon, 8 Jul 2024 12:24:14 -0400 Subject: [PATCH 3/6] update title --- source/developer/aarch64.rst | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/source/developer/aarch64.rst b/source/developer/aarch64.rst index 220d348..9035335 100644 --- a/source/developer/aarch64.rst +++ b/source/developer/aarch64.rst @@ -1,4 +1,4 @@ -``aarch64`` builds +``aarch64`` and ``arm64``` builds ================== .. datechanged:: 2024-03-04 From 51d4e06448fb328d1c65f52f0cf15c80927047fc Mon Sep 17 00:00:00 2001 From: aliciaaevans Date: Mon, 8 Jul 2024 12:30:01 -0400 Subject: [PATCH 4/6] fix formatting --- source/developer/ci-inventory.rst | 4 ++-- 1 file changed, 2 insertions(+), 2 deletions(-) diff --git a/source/developer/ci-inventory.rst b/source/developer/ci-inventory.rst index d223e8b..cb6ff77 100644 --- a/source/developer/ci-inventory.rst +++ b/source/developer/ci-inventory.rst @@ -96,9 +96,9 @@ maintain flexibility over the long term in case a service becomes unusable. - Azure Pipelines, CircleCI - ``bioconda-recipes`` - push to master (bioconda-recipes) - - `azure-pipeline-master.yml `; + - `azure-pipeline-master.yml `_ (``linux-64``, ``osx-64``); `config.yml `_ (``linux-aarch64``, ``osx-arm64``) - - ``linux-64``, ``osx-64``, ``linux-aarch64``, ``osx-arm64` + - ``linux-64``, ``osx-64``, ``linux-aarch64``, ``osx-arm64`` - Runs when a PR is merged to the master branch. The already-built artifacts are retrieved from the PR and uploaded to the ``bioconda`` channel. Container(s) are uploaded to quay.io/biocontainers. If artifacts are not found, it will build the recipe(s) as a fallback. From 750021f46dd3ed6522067852796dcf632321861a Mon Sep 17 00:00:00 2001 From: aliciaaevans Date: Mon, 8 Jul 2024 12:32:08 -0400 Subject: [PATCH 5/6] format to avoid wrapping --- source/developer/repodata_patching.rst | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/source/developer/repodata_patching.rst b/source/developer/repodata_patching.rst index 7851fa8..5944076 100644 --- a/source/developer/repodata_patching.rst +++ b/source/developer/repodata_patching.rst @@ -21,7 +21,7 @@ What is repodata? ----------------- Repodata is a JSON file that contains a variety of information for each package -in Bioconda. There is one for each architecture (linux-64, linux-aarch64, osx-64, osx-arm64, noarch, etc.) +in Bioconda. There is one for each architecture (``linux-64``, ``linux-aarch64``, ``osx-64``, ``osx-arm64``, ``noarch``, etc.) and they're hosted in the bioconda channel. For example, the noarch repodata.json file `is available here `_. Let's take a look at the what sorts of things are stored within this file for a From 9145f0e6fdedbd879db571b163276e57e036bad0 Mon Sep 17 00:00:00 2001 From: aliciaaevans Date: Mon, 8 Jul 2024 12:41:29 -0400 Subject: [PATCH 6/6] remove extra backtick --- source/developer/aarch64.rst | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/source/developer/aarch64.rst b/source/developer/aarch64.rst index 9035335..bbbd86d 100644 --- a/source/developer/aarch64.rst +++ b/source/developer/aarch64.rst @@ -1,4 +1,4 @@ -``aarch64`` and ``arm64``` builds +``aarch64`` and ``arm64`` builds ================== .. datechanged:: 2024-03-04