Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

OSX-Arm64 updates #24

Merged
merged 6 commits into from
Jul 10, 2024
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
17 changes: 10 additions & 7 deletions source/developer/aarch64.rst
Original file line number Diff line number Diff line change
@@ -1,28 +1,31 @@
``aarch64`` builds
``aarch64`` and ``arm64`` builds
==================

.. datechanged:: 2024-03-04
Added this section as initial linux-aarch64 builds are starting

While we do not yet have builds for ``osx-arm64`` (see
:ref:`platform-nomenclature-faq`), we are starting to roll out
``linux-aarch64`` builds. These run on CircleCI, which offers the
infrastructure for this.
.. datechanged:: 2024-04-04
Information about osx-arm64 builds

We are starting to roll out ``linux-aarch64`` and ``osx-arm64`` (see
:ref:`platform-nomenclature-faq`) on CircleCI and GitHub Actions, respectively.
These additional CI platforms offer the runner types needed for these architectures.

This is being initially approached as an opt-in process as we make sure
all the moving parts are working correctly. A recipe can be flagged for
building on ``linux-aarch64`` by adding the following to the
building on ``linux-aarch64`` and/or ``osx-arm64`` by adding the following to the
:file:`meta.yaml` file:

.. code-block:: yaml

extra:
additional-platforms:
- linux-aarch64
- osx-arm64

The current CircleCI config will only run if at least one recipe in the
commit range (typically for the PR) includes the above additional
platform.

Support for building and uploading ``linux-aarch64`` containers is planned but
not yet implemented (as of March 2024).
not yet implemented (as of July 2024).
2 changes: 1 addition & 1 deletion source/developer/bulk.rst
Original file line number Diff line number Diff line change
Expand Up @@ -232,7 +232,7 @@ Some unordered notes on working with the bulk branch:
that don't need to be rebuilt, but that work needs to be done simply to
figure out if a rebuild is needed, and so this is expected.

- For ``linux-64`` and ``osx-64``, the bulk runs take place on GitHub Actions,
- For ``linux-64``, ``osx-64``, and ``osx-arm64`` the bulk runs take place on GitHub Actions,
and the configuration is in :file:`.github/workflows/Bulk.yml`. For
``linux-aarch64``, the builds take place on CircleCI and the configuration is
in :file:`.circleci/config.yml`.
24 changes: 14 additions & 10 deletions source/developer/ci-inventory.rst
Original file line number Diff line number Diff line change
Expand Up @@ -12,6 +12,9 @@ CI Inventory
.. datechanged:: 2024-03-04
Start using ``linux-aarch64`` in bulk and recipe tests

.. datechanged:: 2024-07-05
Start using ``osx-arm64`` in bulk and recipe tests

This page documents the various moving parts that, together, make Bioconda
work. We rely on a mixture of free services to spread the workload and to
maintain flexibility over the long term in case a service becomes unusable.
Expand Down Expand Up @@ -70,8 +73,8 @@ maintain flexibility over the long term in case a service becomes unusable.
- ``bioconda-recipes``
- on push
- `azure-pipeline.yml <https://github.com/bioconda/bioconda-recipes/blob/master/azure-pipeline.yml>`_ (``linux-64``, ``osx-64``);
`config.yml <https://github.com/bioconda/bioconda-recipes/blob/master/.circleci/config.yml>`_ (``linux-aarch64``)
- ``linux-64``, ``osx-64``, ``linux-aarch64``
`config.yml <https://github.com/bioconda/bioconda-recipes/blob/master/.circleci/config.yml>`_ (``linux-aarch64``, ``osx-arm64``)
- ``linux-64``, ``osx-64``, ``linux-aarch64``, ``osx-arm64``
- These are the most-run tests: these are what run on every change on
pull requests to bioconda-recipes, and they must pass before the recipe
is merged into the master branch.
Expand All @@ -90,14 +93,15 @@ maintain flexibility over the long term in case a service becomes unusable.


* - Master branch tests
- Azure Pipelines
- Azure Pipelines, CircleCI
- ``bioconda-recipes``
- push to master (bioconda-recipes)
- `azure-pipeline-master.yml <https://github.com/bioconda/bioconda-recipes/blob/master/azure-pipeline-master.yml>`_
- ``linux-64``, ``osx-64``, ``linux-aarch64``
- Runs the same tests as for PRs, but on the master branch. Note that
when the bot is doing the merge, it adds ``[ci skip]`` in the commit
comment, and works with the already-built artifacts from the PR.
- `azure-pipeline-master.yml <https://github.com/bioconda/bioconda-recipes/blob/master/azure-pipeline-master.yml>`_ (``linux-64``, ``osx-64``);
`config.yml <https://github.com/bioconda/bioconda-recipes/blob/master/.circleci/config.yml>`_ (``linux-aarch64``, ``osx-arm64``)
- ``linux-64``, ``osx-64``, ``linux-aarch64``, ``osx-arm64``
- Runs when a PR is merged to the master branch. The already-built artifacts are retrieved from the PR and uploaded to the ``bioconda`` channel.
Container(s) are uploaded to quay.io/biocontainers.
If artifacts are not found, it will build the recipe(s) as a fallback.


* - Mergify
Expand Down Expand Up @@ -210,9 +214,9 @@ maintain flexibility over the long term in case a service becomes unusable.
- GitHub Actions
- ``bioconda-recipes``
- push to ``bulk`` branch of bioconda-utils
- `Bulk.yml <https://github.com/bioconda/bioconda-recipes/blob/master/.github/workflows/Bulk.yml>`_ (``linux-64``, ``osx-64``);
- `Bulk.yml <https://github.com/bioconda/bioconda-recipes/blob/master/.github/workflows/Bulk.yml>`_ (``linux-64``, ``osx-64``, ``osx-arm64``);
`config.yml <https://github.com/bioconda/bioconda-recipes/blob/master/.circleci/config.yml>`_ (``linux-aarch64``)
- ``linux-64``, ``osx-64``, ``linux-aarch64``
- ``linux-64``, ``osx-64``, ``linux-aarch64``, ``osx-arm64``
- If pushing to the special ``bulk`` branch, this workflow will run. It
uses special bioconda-utils functionality to split the full DAG into
sub-DAGs and submits them to independent parallel jobs. When recipes
Expand Down
2 changes: 1 addition & 1 deletion source/developer/repodata_patching.rst
Original file line number Diff line number Diff line change
Expand Up @@ -21,7 +21,7 @@ What is repodata?
-----------------

Repodata is a JSON file that contains a variety of information for each package
in Bioconda. There is one for each architecture (linux-64, linux-aarch64, osx-64, noarch, etc.)
in Bioconda. There is one for each architecture (``linux-64``, ``linux-aarch64``, ``osx-64``, ``osx-arm64``, ``noarch``, etc.)
and they're hosted in the bioconda channel. For example, the noarch repodata.json file `is
available here <https://conda.anaconda.org/bioconda/noarch/repodata.json>`_.
Let's take a look at the what sorts of things are stored within this file for a
Expand Down
3 changes: 1 addition & 2 deletions source/faqs.rst
Original file line number Diff line number Diff line change
Expand Up @@ -82,8 +82,7 @@ What versions are supported?

**Operating Systems:**
Bioconda supports Linux (x86_64 and aarch64/arm64) and
macOS (x86_64). ARM is not currently supported for macOS. Windows is not
supported.
macOS (x86_64 and arm64). Windows is not supported.

**Python:**
Bioconda currently supports Python 3.8, 3.9, and 3.10 (see "pinned packages"
Expand Down
2 changes: 1 addition & 1 deletion source/index.rst
Original file line number Diff line number Diff line change
Expand Up @@ -20,7 +20,7 @@
**Bioconda** lets you install thousands of software packages related to
biomedical research using the `conda <https://conda.io>`_ package manager.

**NOTE**: *Bioconda supports only Linux (64-bit and AArch64) and macOS (x86_64)*
**NOTE**: *Bioconda supports only Linux (64-bit and AArch64) and macOS (x86_64 and ARM64)*

Usage
=====
Expand Down