Hermetic build of cachi2 image using Docker multi-stage building #527
Conversation
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
eskultety
changed the title
This reverts commit 6d8b0be. A future patch will switch the container image from using Fedora 39 to using UBI9 (Red Hat Universal Base Image 9). Since that one ships with a pretty old version of Nodejs (v16.20.2) we need to revert this Dependabot commit to get original the engine dep rule back. Signed-off-by: Erik Skultety <[email protected]>
We don't require NPM for anything really, we only need a version that supplies corepack which we'll in turn use to install a specific version of Yarn. Given that we're going to switch the container image from Fedora to UBI9 in a future patch which only ships with an old NodeJS v16.30.2 (i.e. one officially marked as in maintenance from upstream perspective) we need to prevent dependabot from trying to submit update PRs for it. This can be easily done by providing an upper boundary to the version of the NodeJS engine. Signed-off-by: Erik Skultety <[email protected]>
Commit 9967a2e forgot to update the commentary when adding another fixed version of Go SDK to download and install during the image build. Fixes: 9967a2e Signed-off-by: Erik Skultety <[email protected]>
We're specifically interested in the 'go' binary which is part of the compiler toolkit. It'll pull in 'golang' anyway, so there's no practical difference from the build's POV, it's the slight nuance of what we actually require for the resulting container and being exact in terms of listing it. Signed-off-by: Erik Skultety <[email protected]>
We may need this in order to build some of our Python dependencies (e.g. reflink) that don't build as a wheel (because they're heavily platform dependent) and need to be built locally during installation. Signed-off-by: Erik Skultety <[email protected]>
Red Hat ships UBIs as free images with the same stability and security guarantees like any other Red Hat product, so in terms of the runtime environment for cachi2 it makes sense to use a stable, curated, and well maintained base image. As a side-effect, this will rid of the need to follow Fedora releases and bump the base image version every 6 months or so when the next major version is releases, marking the 2nd oldest major release as EOL. However, using a stable and well maintained image isn't rainbows and unicorns either as we'll likely fall into the trap of lacking the latest package manager tooling required by cachi2 consumers which Fedora kinda solved (e.g. by always using the latest release to get the latest Golang version with the exception of >=1.21 and toolchains), but not without any caveats, those would just become more apparent with the selection of the next base image. Since we're switching base images, this patch also updates the integration mock data based on what's reported on UBI9. Resolves: containerbuildsystem#401 Signed-off-by: Erik Skultety <[email protected]>
|
Latest changes:
Note that this PR will remain in a draft for a while and the latest revision won't likely build hermetically due to reflink, I just want to see if the CI is going to be happy with the changes to progress further from there. Nevertheless, comments are always welcome! |
- reduce image size roughly by half - prepare the project to be built hermetically - Debian Bullseye was selected for the Golang base image instead of Alpine because Alpine links the go binary with musl C implementation instead of glibc (we'd have to install musl into the final image) - update test bom.json (some modules are missing because we don't install platform Go anymore) Signed-off-by: Erik Skultety <[email protected]>
This was just a band aid over the fact that we needed corepack and yarn installed correctly. With multi-stage container build this is no longer needed as both corepack and yarn are copied over from the official Node Alpine images. Signed-off-by: Erik Skultety <[email protected]>
Signed-off-by: Erik Skultety <[email protected]>
In order to build cachi2 hermetically we'll need setuptools-scm and wheel in order to process our Python build requirements in form of wheels (sdists are a complete no-go!). Signed-off-by: Erik Skultety <[email protected]>
Based on UBI9 we'll need a lockfile to be able to prefetch all the RPMs for cachi2's own hermetic build. Signed-off-by: Erik Skultety <[email protected]>
|
Some of the work was split into individual PRs - see also: |
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
Draft PR on a what is to become a hermetic build of cachi2 itself based on the multi-stage Dockerfile feature where we copy over the most problematic bits we could not fetch in a convenient way otherwise: Go SDKs, NodeJS's corepack and Yarn.
Maintainers will complete the following section
Note: if the contribution is external (not from an organization member), the CI
pipeline will not run automatically. After verifying that the CI is safe to run:
/ok-to-test(as is the standard for Pipelines as Code)