Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

expose build args, format file, update deps and fix diggest upload #44

Merged
merged 1 commit into from
Oct 31, 2024
Merged

expose build args, format file, update deps and fix diggest upload #44

Changes from all commits
Commits
Show all changes
1 commit
Select commit
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
115 changes: 57 additions & 58 deletions .github/workflows/build.yaml
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -5,80 +5,78 @@ on:
region:
required: false
type: string
default: 'us-east-1'
description: 'AWS region'
default: "us-east-1"
description: "AWS region"
tags:
required: true
type: string
description: 'Docker build tags'
description: "Docker build tags"
registry:
required: true
type: string
description: 'Registry image name'
description: "Registry image name"
dockerfile_path:
required: false
type: string
default: build/Dockerfile
description: 'Dockerfile relative path'
description: "Dockerfile relative path"
docker_target:
required: false
type: string
default: ''
description: 'Dockerfile target'
default: ""
description: "Dockerfile target"
runner:
required: false
type: string
default: 'non-prod'
description: 'Runner type'
default: "non-prod"
description: "Runner type"
multiplatform:
required: false
type: boolean
default: false
description: 'Multiplatform build'

description: "Multiplatform build"
secrets:
API_TOKEN_GITHUB:
required: true
description: 'Github token hash'
description: "Github token hash"
AWS_ACCESS_KEY_ID:
required: true
description: 'AWS access key id'
description: "AWS access key id"
AWS_SECRET_ACCESS_KEY:
required: true
description: 'AWS secret access key'

description: "AWS secret access key"
jobs:
release:
name: Build Docker
if: ${{ ! inputs.multiplatform }}
runs-on: ${{ inputs.runner }}
steps:
- name: Setup | Checkout
uses: actions/checkout@v4
with:
submodules: true
token: ${{ secrets.API_TOKEN_GITHUB }}

- name: Docker | Build and Push
uses: timescale/cloud-actions/build-push@main
with:
aws-access-key-id: ${{ secrets.AWS_ACCESS_KEY_ID }}
aws-secret-access-key: ${{ secrets.AWS_SECRET_ACCESS_KEY }}
region: ${{ inputs.region }}
tags: |
${{ inputs.tags }}
registry: ${{ inputs.registry }}
target: ${{ inputs.docker_target }}
file: ${{ inputs.dockerfile_path }}
- name: Setup | Checkout
uses: actions/checkout@v4
with:
submodules: true
token: ${{ secrets.API_TOKEN_GITHUB }}

- name: Docker | Build and Push
uses: timescale/cloud-actions/build-push@main
with:
aws-access-key-id: ${{ secrets.AWS_ACCESS_KEY_ID }}
aws-secret-access-key: ${{ secrets.AWS_SECRET_ACCESS_KEY }}
region: ${{ inputs.region }}
tags: |
${{ inputs.tags }}
registry: ${{ inputs.registry }}
target: ${{ inputs.docker_target }}
file: ${{ inputs.dockerfile_path }}

mp-build:
name: Build multiplatform Image (${{ matrix.platform }})
if: ${{ inputs.multiplatform }}
runs-on: [ "self-hosted", "${{ matrix.runs_on }}" ]
runs-on: ["self-hosted", "${{ matrix.runs_on }}"]
strategy:
fail-fast: false
matrix:
platform: [ "amd64", "arm64" ]
platform: ["amd64", "arm64"]
include:
- platform: amd64
runs_on: dev-us-east-1
Expand All @@ -87,11 +85,6 @@ jobs:
steps:
- name: Checkout code
uses: actions/checkout@v4

- name: Prepare
run: |
platform_slash_pair=linux/${{ matrix.platform }}
echo "PLATFORM_DASH_PAIR=${platform_slash_pair//\//-}" >> $GITHUB_ENV

- name: Docker meta
id: meta
Expand All @@ -100,15 +93,15 @@ jobs:
images: ${{ inputs.registry }}
tags: |
type=raw,value=${{ inputs.tags }}

- name: Set up QEMU
uses: docker/setup-qemu-action@v3

- name: Set up Docker Buildx
uses: docker/setup-buildx-action@v3

- name: Configure AWS credentials
uses: aws-actions/configure-aws-credentials@v1
uses: aws-actions/configure-aws-credentials@v4
with:
aws-access-key-id: ${{ secrets.AWS_ACCESS_KEY_ID }}
aws-secret-access-key: ${{ secrets.AWS_SECRET_ACCESS_KEY }}
Expand All @@ -123,24 +116,30 @@ jobs:
with:
context: .
platforms: linux/${{ matrix.platform }}
target: ${{ inputs.docker_target }}
labels: ${{ steps.meta.outputs.labels }}
outputs: type=image,name=${{ inputs.registry }},push-by-digest=true,name-canonical=true,push=true

build-args: |
GOOS=linux
GOARCH=${{ matrix.platform }}
BPF_TARGET=${{ matrix.platform }}

- name: Export digest
run: |
mkdir -p /tmp/digests
rm -rf /tmp/digests/*
mkdir -p /tmp/digests
digest="${{ steps.build.outputs.digest }}"
touch "/tmp/digests/${digest#sha256:}"

export "IMAGE_DIGEST=${digest#sha256:}"
echo "IMAGE_DIGEST=${IMAGE_DIGEST}" >> $GITHUB_ENV
touch "/tmp/digests/${IMAGE_DIGEST}"

- name: Upload digest
uses: actions/upload-artifact@v4
with:
name: digests-${{ env.PLATFORM_DASH_PAIR }}
name: digests-${{ inputs.docker_target }}-${{ env.IMAGE_DIGEST }}
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The docker target isn't a required argument- what happens if it's not set? Just digests--<digest>? I think the platform is pretty important to include- I'm using depending on it as digests-{{ matrix.platform }}-* in the download steps elsewhere so it can properly attribute the arch. I'd suggest adding something instead of replacing the platform env?

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

in this case it will generate a file like digest--2369058236gjgfhsdjgsjdtgyu23u852365

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

let me update the problem in the read me, when using the current strategy twice (parliament and owl) it fails because the file already exists.

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Yup, that would totally happen! The IMAGE_DIGEST should stop that for sure.

path: /tmp/digests/*
if-no-files-found: error
retention-days: 1

merge:
runs-on: ${{ inputs.runner }}
needs:
Expand All @@ -150,41 +149,41 @@ jobs:
run: |
mkdir -p /tmp/digests
rm -rf /tmp/digests/*

- name: Download digests
uses: actions/download-artifact@v4
with:
path: /tmp/digests
pattern: digests-*
pattern: digests-${{ inputs.docker_target }}-*
merge-multiple: true

- name: Set up Docker Buildx
uses: docker/setup-buildx-action@v3

- name: Docker meta
id: meta
uses: docker/metadata-action@v5
with:
images: ${{ inputs.registry }}
tags: |
type=raw,value=${{ inputs.tags }}

- name: Configure AWS credentials
uses: aws-actions/configure-aws-credentials@v1
uses: aws-actions/configure-aws-credentials@v4
with:
aws-access-key-id: ${{ secrets.AWS_ACCESS_KEY_ID }}
aws-secret-access-key: ${{ secrets.AWS_SECRET_ACCESS_KEY }}
aws-region: ${{ inputs.region }}

- name: Login to Amazon ECR
uses: aws-actions/amazon-ecr-login@v2

- name: Create manifest list and push
working-directory: /tmp/digests
run: |
docker buildx imagetools create $(jq -cr '.tags | map("-t " + .) | join(" ")' <<< "$DOCKER_METADATA_OUTPUT_JSON") \
$(printf '${{ inputs.registry }}@sha256:%s ' *)
$(printf '${{ inputs.registry }}@sha256:%s ' *)

- name: Inspect image
run: |
docker buildx imagetools inspect ${{ inputs.registry }}:${{ inputs.tags }}
Loading