Skip to content

Commit

Permalink
Refactor docker images and workflows to build and deploy them
Browse files Browse the repository at this point in the history 
    * Remove workflow `deploy-build-kit.yaml`. It is integrated into
        `deploy-docker-images.yml`.
    * Refactor deploy-docker-images.yml workflow
        * Convert matrix into multiple jobs, one for each docker image,
        because the images depend on each other, and the matrix strategy
        does not support this.
        * Add `env-setup` job to set up the environment for the other jobs,
        because reusable workflows don't support environment variables.
        * Add input parameter to build deprecated images. Currently, the
        deprecated images are built by default, if not specified otherwise.
        * Use output `one_image_tag_short` as input for the dependent images
    * Refactor build-single-docker-image.yml
        * Rename input `docker_directory` to `directory`. Instead of setting the
        directory that contains all images, it is set now to the directory that
        contains the Dockerfile for the image to build.
        * Add input parameter `docker_file_name` to specify the Dockerfile to use.
        * Add input parameter `platforms` to specify the platforms to build the
        image for. The default is `linux/amd64,linux/arm64` and `linux/arm/v7`.
        * Add input parameter `depends_on_paths` to specify the paths that the
        image depends on. If the paths change, the image is rebuilt.
        * Add input parameter `build_args` to specify the build arguments for the
        image.
        * The name of the deployed image is now set by the input parameter
        `image_name`: `ghcr.io/everest/<image_name>`. The directory name is
        independent of the image name.
        * Add output parameter `rebuild` to indicate if the image was rebuilt.
        * Add output parameter `image_tag_short` to pass the image tag to the
        dependent images.
        * Add output parameter `image_tag_long` to pass the full image tag to the
        dependent images.
        * Add output parameter `image_tags` to pass all full image tags.
        * Fix rebuild conditions
    * Update README.md
    * Deprecate `ghcr.io/everest/everest-clang-format` image. Its features are now included in
        `ghcr.io/everest/everest-ci/build-env-base` The old image is deprecated, but still available and deployed.
    * Merge `ghcr.io/everest/build-kit-alpine` and
        `ghcr.io/everest/build-kit-debian` into `ghcr.io/everest/everest-ci/build-kit-base`.
        The old images are deprecated, but still available and deployed.
    * Add `ghcr.io/everest/everest-ci/run-env-base` image
    * Add `ghcr.io/everest/everest-ci/build-env-base` image
    * Add `ghcr.io/everest/everest-ci/dev-env-base` image
    * Add `run-clang-format.py` script to `build-env-base` image
    * Update github action `run-glang-format` to use new `build-env-base` image

Signed-off-by: Andreas Heinrich <[email protected]>
  • Loading branch information
@andistorm
andistorm committed Jul 16, 2024
1 parent 5829949 commit b7b5581
Show file tree
Hide file tree
Showing 22 changed files with 1,100 additions and 74 deletions.
44 changes: 0 additions & 44 deletions .github/workflows/deploy-build-kit.yaml
View file

This file was deleted.

184 changes: 174 additions & 10 deletions .github/workflows/deploy-docker-images.yml
View file
Original file line number Diff line number Diff line change
Expand Up @@ -7,6 +7,10 @@ on:
description: 'Force rebuild of all images'
default: false
type: boolean
build_deprecated_images:
description: 'Build deprecated images'
default: false
type: boolean
push:
branches:
- '**'
Expand All @@ -15,21 +19,181 @@ on:

env:
REGISTRY: ghcr.io
DOCKER_DIRECTORY: docker/images
PLATFORMS: |
linux/amd64
PATH_TO_DEPLOY_SINGLE_DOCKER_IMAGE_WORKFLOW: .github/workflows/deploy-single-docker-image.yml
PATH_TO_DEPLOY_DOCKER_IMAGES_WORKFLOW: .github/workflows/deploy-docker-images.yml

jobs:
build-and-push-all-images:
name: Build and push all docker images
strategy:
matrix:
image_name: [everest-clang-format]
uses: everest/everest-ci/.github/workflows/[email protected]
env-setup:
# Since env variables can't be passed to reusable workflows, we need to pass them as outputs
name: Evaluate force rebuild and set env variables as outputs
runs-on: ubuntu-22.04
outputs:
force_rebuild: ${{ steps.check.outputs.force_rebuild }}
docker_registry: ${{ env.REGISTRY }}
docker_directory: ${{ env.DOCKER_DIRECTORY }}
platforms: ${{ env.PLATFORMS }}
repository_name: ${{ github.event.repository.name }}
build_deprecated_images: ${{ steps.check.outputs.build_deprecated_images }}
path_to_deploy_single_docker_image_workflow: ${{ env.PATH_TO_DEPLOY_SINGLE_DOCKER_IMAGE_WORKFLOW }}
path_to_deploy_docker_images_workflow: ${{ env.PATH_TO_DEPLOY_DOCKER_IMAGES_WORKFLOW }}
steps:
- id: check
run: |
echo "force_rebuild=${{ inputs.force_rebuild || (github.event_name == 'push' && startsWith(github.event.ref, 'refs/tags/' )) || false }}" >> $GITHUB_OUTPUT
# Build deprecated images if not explicitly disabled
#TODO: set default to false, once backwards compatibility is no longer needed
echo "build_deprecated_images=${{ inputs.build_deprecated_images || true }}" >> $GITHUB_OUTPUT
# One job for each image, since the images build on top of each other a matrix strategy is not possible
run-env-base:
needs:
- env-setup
name: Build and push run-env-base docker image
uses: ./.github/workflows/deploy-single-docker-image.yml
secrets:
SA_GITHUB_PAT: ${{ secrets.SA_GITHUB_PAT }}
SA_GITHUB_USERNAME: ${{ secrets.SA_GITHUB_USERNAME }}
with:
force_rebuild: ${{ needs.env-setup.outputs.force_rebuild == 'true' }}
image_name: ${{ needs.env-setup.outputs.repository_name }}/run-env-base
directory: ${{ needs.env-setup.outputs.docker_directory }}/run-env-base
docker_registry: ${{ needs.env-setup.outputs.docker_registry }}
github_ref_before: ${{ github.event.before }}
github_ref_after: ${{ github.event.after }}
platforms: ${{ needs.env-setup.outputs.platforms }}
depends_on_paths: |
${{ needs.env-setup.outputs.path_to_deploy_single_docker_image_workflow }}
${{ needs.env-setup.outputs.path_to_deploy_docker_images_workflow }}
build-env-base:
needs:
- env-setup
- run-env-base
name: Build and push build-env-base docker image
uses: ./.github/workflows/deploy-single-docker-image.yml
secrets:
SA_GITHUB_PAT: ${{ secrets.SA_GITHUB_PAT }}
SA_GITHUB_USERNAME: ${{ secrets.SA_GITHUB_USERNAME }}
with:
force_rebuild: ${{ needs.env-setup.outputs.force_rebuild == 'true' || needs.run-env-base.outputs.rebuild == 'true' }}
image_name: ${{ needs.env-setup.outputs.repository_name }}/build-env-base
directory: ${{ needs.env-setup.outputs.docker_directory }}/build-env-base
docker_registry: ${{ needs.env-setup.outputs.docker_registry }}
github_ref_before: ${{ github.event.before }}
github_ref_after: ${{ github.event.after }}
platforms: ${{ needs.env-setup.outputs.platforms }}
depends_on_paths: |
${{ needs.env-setup.outputs.path_to_deploy_single_docker_image_workflow }}
${{ needs.env-setup.outputs.path_to_deploy_docker_images_workflow }}
build_args: |
BASE_IMAGE_TAG=${{ needs.run-env-base.outputs.one_image_tag_short }}
dev-env-base:
needs:
- env-setup
- build-env-base
name: Build and push dev-env-base docker image
uses: ./.github/workflows/deploy-single-docker-image.yml
secrets:
SA_GITHUB_PAT: ${{ secrets.SA_GITHUB_PAT }}
SA_GITHUB_USERNAME: ${{ secrets.SA_GITHUB_USERNAME }}
with:
force_rebuild: ${{ inputs.force_rebuild || (github.event_name == 'push' && startsWith(github.event.ref, 'refs/tags/' )) || false }}
image_name: ${{ matrix.image_name }}
docker_directory: docker/images/
docker_registry: ghcr.io
force_rebuild: ${{ needs.env-setup.outputs.force_rebuild == 'true' || needs.build-env-base.outputs.rebuild == 'true' }}
image_name: ${{ needs.env-setup.outputs.repository_name }}/dev-env-base
directory: ${{ needs.env-setup.outputs.docker_directory }}/dev-env-base
docker_registry: ${{ needs.env-setup.outputs.docker_registry }}
github_ref_before: ${{ github.event.before }}
github_ref_after: ${{ github.event.after }}
platforms: ${{ needs.env-setup.outputs.platforms }}
depends_on_paths: |
${{ needs.env-setup.outputs.path_to_deploy_single_docker_image_workflow }}
${{ needs.env-setup.outputs.path_to_deploy_docker_images_workflow }}
build_args: |
BASE_IMAGE_TAG=${{ needs.build-env-base.outputs.one_image_tag_short }}
build-kit-base:
needs:
- env-setup
- build-env-base
name: Build and push build-kit docker image
uses: ./.github/workflows/deploy-single-docker-image.yml
secrets:
SA_GITHUB_PAT: ${{ secrets.SA_GITHUB_PAT }}
SA_GITHUB_USERNAME: ${{ secrets.SA_GITHUB_USERNAME }}
with:
force_rebuild: ${{ needs.env-setup.outputs.force_rebuild == 'true' || needs.build-env-base.outputs.rebuild == 'true' }}
image_name: ${{ needs.env-setup.outputs.repository_name }}/build-kit-base
directory: ${{ needs.env-setup.outputs.docker_directory }}/build-kit-base
docker_registry: ${{ needs.env-setup.outputs.docker_registry }}
github_ref_before: ${{ github.event.before }}
github_ref_after: ${{ github.event.after }}
platforms: ${{ needs.env-setup.outputs.platforms }}
depends_on_paths: |
${{ needs.env-setup.outputs.path_to_deploy_single_docker_image_workflow }}
${{ needs.env-setup.outputs.path_to_deploy_docker_images_workflow }}
build_args: |
BASE_IMAGE_TAG=${{ needs.build-env-base.outputs.one_image_tag_short }}
# Include deprecated images for backwards compatibility
deprecated-everest-clang-format:
if: ${{ needs.env-setup.outputs.build_deprecated_images == 'true' }}
needs:
- env-setup
name: Build and push deprecated everest-clang-format docker image
uses: ./.github/workflows/deploy-single-docker-image.yml
secrets:
SA_GITHUB_PAT: ${{ secrets.SA_GITHUB_PAT }}
SA_GITHUB_USERNAME: ${{ secrets.SA_GITHUB_USERNAME }}
with:
force_rebuild: ${{ needs.env-setup.outputs.force_rebuild == 'true' }}
image_name: everest-clang-format
directory: docker/deprecated-images/everest-clang-format
docker_registry: ${{ needs.env-setup.outputs.docker_registry }}
github_ref_before: ${{ github.event.before }}
github_ref_after: ${{ github.event.after }}
platforms: ${{ needs.env-setup.outputs.platforms }}
depends_on_paths: |
${{ needs.env-setup.outputs.path_to_deploy_single_docker_image_workflow }}
${{ needs.env-setup.outputs.path_to_deploy_docker_images_workflow }}
deprecated-build-kit-alpine:
if: ${{ needs.env-setup.outputs.build_deprecated_images == 'true' }}
needs:
- env-setup
name: Build and push deprecated build-kit-alpine docker image
uses: ./.github/workflows/deploy-single-docker-image.yml
secrets:
SA_GITHUB_PAT: ${{ secrets.SA_GITHUB_PAT }}
SA_GITHUB_USERNAME: ${{ secrets.SA_GITHUB_USERNAME }}
with:
force_rebuild: ${{ needs.env-setup.outputs.force_rebuild == 'true' }}
image_name: build-kit-alpine
directory: docker/deprecated-images/build-kit
docker_file_name: alpine.Dockerfile
docker_registry: ${{ needs.env-setup.outputs.docker_registry }}
github_ref_before: ${{ github.event.before }}
github_ref_after: ${{ github.event.after }}
platforms: ${{ needs.env-setup.outputs.platforms }}
depends_on_paths: |
${{ needs.env-setup.outputs.path_to_deploy_single_docker_image_workflow }}
${{ needs.env-setup.outputs.path_to_deploy_docker_images_workflow }}
deprecated-build-kit-debian:
if: ${{ needs.env-setup.outputs.build_deprecated_images == 'true' }}
needs:
- env-setup
name: Build and push deprecated build-kit-debian docker image
uses: ./.github/workflows/deploy-single-docker-image.yml
secrets:
SA_GITHUB_PAT: ${{ secrets.SA_GITHUB_PAT }}
SA_GITHUB_USERNAME: ${{ secrets.SA_GITHUB_USERNAME }}
with:
force_rebuild: ${{ needs.env-setup.outputs.force_rebuild == 'true' }}
image_name: build-kit-debian
directory: docker/deprecated-images/build-kit
docker_file_name: debian.Dockerfile
docker_registry: ${{ needs.env-setup.outputs.docker_registry }}
github_ref_before: ${{ github.event.before }}
github_ref_after: ${{ github.event.after }}
platforms: ${{ needs.env-setup.outputs.platforms }}
depends_on_paths: |
${{ needs.env-setup.outputs.path_to_deploy_single_docker_image_workflow }}
${{ needs.env-setup.outputs.path_to_deploy_docker_images_workflow }}
Loading

0 comments on commit b7b5581

Please sign in to comment.