Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

✨Infra: Build for arm64 #6984

Open
wants to merge 17 commits into
base: master
Choose a base branch
from

Conversation

sanderegg
Copy link
Member

@sanderegg sanderegg commented Dec 19, 2024

What do these changes do?

In order to run oSparc on ARM-based processors, this PR allows to build docker images for ARM64 processors
This is useful for cost reduction purposes.
re-used Makefile machinery used for building towards Raspberry Pi ARM architecture.

1st step shall run manually via triggering. next steps will follow in a separate PR.

some details:

  • Dockerfile TARGETARCH variable contains the target architecture (automatically filled in by docker at build time) (see Dockerfile multi-architecture docs)
  • Already adapted RClone installation

Bonuses:

  • upgraded shellcheck to 0.10.0

Current workflow

  • first create the usual amd64 image, and run all the tests as usual
  • once this goes through and in master,

Currently will be run manually for testing purposes

  • the arm64 images will be built (sadly for now with possibly slightly different libs as the build for arm64 happens at a different timepoint),
  • the arm64 build is pushed to dockerhub,
  • the arm64 and amd64 images are then "fused" as the registry is able to provide both architectures at the same time

Related issue/s

How to test

Dev-ops checklist

@sanderegg sanderegg added the a:infra+ops maintenance of infrastructure or operations (discussed in retro) label Dec 19, 2024
@sanderegg sanderegg added this to the Event Horizon milestone Dec 19, 2024
@sanderegg sanderegg self-assigned this Dec 19, 2024
@sanderegg sanderegg marked this pull request as draft December 19, 2024 16:31
Copy link

codecov bot commented Dec 19, 2024

Codecov Report

All modified and coverable lines are covered by tests ✅

Project coverage is 81.49%. Comparing base (9d5c8db) to head (1f7b717).

Additional details and impacted files
@@            Coverage Diff             @@
##           master    #6984      +/-   ##
==========================================
- Coverage   85.52%   81.49%   -4.03%     
==========================================
  Files        1630      120    -1510     
  Lines       64379     5070   -59309     
  Branches     2035        0    -2035     
==========================================
- Hits        55061     4132   -50929     
+ Misses       8984      938    -8046     
+ Partials      334        0     -334     
Flag Coverage Δ
integrationtests 59.86% <ø> (-5.02%) ⬇️
unittests 88.18% <ø> (+4.37%) ⬆️
Components Coverage Δ
api ∅ <ø> (∅)
pkg_aws_library ∅ <ø> (∅)
pkg_dask_task_models_library ∅ <ø> (∅)
pkg_models_library ∅ <ø> (∅)
pkg_notifications_library ∅ <ø> (∅)
pkg_postgres_database ∅ <ø> (∅)
pkg_service_integration ∅ <ø> (∅)
pkg_service_library ∅ <ø> (∅)
pkg_settings_library ∅ <ø> (∅)
pkg_simcore_sdk ∅ <ø> (∅)
agent 96.82% <ø> (ø)
api_server ∅ <ø> (∅)
autoscaling ∅ <ø> (∅)
catalog ∅ <ø> (∅)
clusters_keeper ∅ <ø> (∅)
dask_sidecar ∅ <ø> (∅)
datcore_adapter ∅ <ø> (∅)
director ∅ <ø> (∅)
director_v2 ∅ <ø> (∅)
dynamic_scheduler ∅ <ø> (∅)
dynamic_sidecar 89.75% <ø> (ø)
efs_guardian ∅ <ø> (∅)
invitations ∅ <ø> (∅)
osparc_gateway_server ∅ <ø> (∅)
payments ∅ <ø> (∅)
resource_usage_tracker ∅ <ø> (∅)
storage ∅ <ø> (∅)
webclient ∅ <ø> (∅)
webserver 56.27% <ø> (-23.45%) ⬇️

Continue to review full report in Codecov by Sentry.

Legend - Click here to learn more
Δ = absolute <relative> (impact), ø = not affected, ? = missing data
Powered by Codecov. Last update 9d5c8db...1f7b717. Read the comment docs.

Copy link
Member

@pcrespov pcrespov left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

i wonder whether we should also have some of those for development

Copy link
Contributor

@GitHK GitHK left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

reassign when ready

@sanderegg sanderegg force-pushed the build-for-arm64 branch 3 times, most recently from 02015ca to 86fb6ec Compare January 6, 2025 14:11
@sanderegg sanderegg marked this pull request as ready for review January 6, 2025 14:12
Copy link
Contributor

@matusdrobuliak66 matusdrobuliak66 left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

👍

Copy link
Member

@pcrespov pcrespov left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

thx

.github/workflows/ci-arm-build.yml Show resolved Hide resolved
.github/workflows/ci-arm-build.yml Show resolved Hide resolved
.github/workflows/ci-arm-build.yml Show resolved Hide resolved
Copy link
Member

@mrnicegyu11 mrnicegyu11 left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

lgtm keep us in the loop on the progress, thx

@sanderegg sanderegg force-pushed the build-for-arm64 branch 3 times, most recently from 64389f1 to cb6a301 Compare January 7, 2025 08:09
Copy link

sonarqubecloud bot commented Jan 7, 2025

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
a:infra+ops maintenance of infrastructure or operations (discussed in retro)
Projects
None yet
Development

Successfully merging this pull request may close these issues.

6 participants