Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Bulk arm builds #43995

Closed
wants to merge 123 commits into from
Closed

Bulk arm builds #43995

wants to merge 123 commits into from

Conversation

aliciaaevans
Copy link
Contributor

@aliciaaevans aliciaaevans commented Oct 31, 2023

Port .github/workflows/bulk.yml to Circle CI for ARM builds.

To Do:

  • determine what's needed to commit as bioconda-bot
  • test

Edited by @daler 2023-11-26. Note to future readers: this PR has become a sort of sandbox for all things ARM. There is no intention that this should ever be merged. Instead, working parts should be cherry-picked or manually ported over to the arm-builds branch, #40550

johanneskoester and others added 30 commits April 21, 2023 16:13
Co-authored-by: Yikun Jiang <[email protected]>
Co-authored-by: Yikun Jiang <[email protected]>
Co-authored-by: Yikun Jiang <[email protected]>
Co-authored-by: Yikun Jiang <[email protected]>
Co-authored-by: Yikun Jiang <[email protected]>
Co-authored-by: Yikun Jiang <[email protected]>
Co-authored-by: Yikun Jiang <[email protected]>
* abort job if platform not found

* sudo for yq call

* debug yq error

* separate regex

* more debugging

* check yq version

* more debugging

* even more debugging

* check shell

* more debugging

* isolate debug checks

* trial and error

* test

* download correct yq version

* check arch of yq

* Download ARM build of yq

* Add platform to recipe

* dynamically get yq platform

* test skip

* move check to script

* permissions

* swap argument order
@daler
Copy link
Member

daler commented Nov 26, 2023

@bgruening got it working, see bioconda/bioconda-utils#941.

That allowed the container to upload, BUT it was set to private. I figured out I needed to run the change visibility GitHub Action workflow over on bioconda-utils to get it public. After that, I was able to successfully run

docker run --platform linux/arm64 -it quay.io/biocontainers/test:0.1--h73052cd_7

🎉

Only thing I can think of that would cause it to stay private is that the CircleCI token needs to be updated. I don't have access to biocontainers on quay.io though.

Can you verify that the env vars in the CircleCI settings for bioconda-recipes are up to date?

bgruening added a commit to bioconda/bioconda-utils that referenced this pull request Nov 26, 2023
Port the fixes in f482801, which were
in `pkg_test.py`, to `upload.py` as well.

ping @bgruening this is what was causing that strangely different
behavior in `mulled-build`'s `build-and-test` and `push` over in
bioconda/bioconda-recipes#43995.

After reviewing this and tracking down all the various moving parts,
it's *really* unclear how the various docker images are specified
throughout all our machinery. Explicit and implicit env vars set at
various times, sometimes overridden by default args, sometimes not,
sometimes hard-coded deep in bioconda-utils. Needs a refactor, where
everything gets specified just once in bioconda-common and used
everywhere. That's for another day though...

---------

Co-authored-by: Björn Grüning <[email protected]>
@daler
Copy link
Member

daler commented Nov 26, 2023

@BiocondaBot please fetch artifacts

@BiocondaBot
Copy link
Collaborator

No artifacts found on the most recent Azure build. Either the build failed, the artifacts have were removed due to age, or the recipe was blacklisted/skipped.

@daler
Copy link
Member

daler commented Nov 26, 2023

So that fetching failed; looking more closely, the bot will need quite a bit more work to support circleci artifacts. However this is implemented in bioconda-utils with handle-merged-pr, so I'll try using the test recipe in a separate PR to see if everything but the bot's artifact fetching works.

@aliciaaevans aliciaaevans mentioned this pull request Nov 30, 2023
Base automatically changed from arm-builds to master March 4, 2024 21:42
@aliciaaevans
Copy link
Contributor Author

Now that #40550 has been merged, closing this.

@aliciaaevans aliciaaevans deleted the bulk-arm-builds branch March 6, 2024 17:16
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
Projects
None yet
Development

Successfully merging this pull request may close these issues.

7 participants