Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

ci: always build guix, save artifacts #6197

Merged
merged 4 commits into from
Aug 12, 2024
Merged
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
98 changes: 67 additions & 31 deletions .github/workflows/guix-build.yml
Original file line number Diff line number Diff line change
@@ -1,15 +1,18 @@
name: Guix Build

permissions:
packages: write

on:
pull_request:
types: [ labeled ]
workflow_dispatch:
pull_request_target:
push:

jobs:
build:
runs-on: [ "self-hosted", "linux", "x64", "ubuntu-core" ]
if: contains(github.event.pull_request.labels.*.name, 'guix-build')
timeout-minutes: 480
build-image:
runs-on: ubuntu-latest
outputs:
image-tag: ${{ steps.prepare.outputs.image-tag }}
repo-name: ${{ steps.prepare.outputs.repo-name }}
steps:
- name: Checkout
uses: actions/checkout@v4
Expand All @@ -22,37 +25,72 @@ jobs:
uses: docker/setup-buildx-action@v3

- name: Commit variables
id: dockerfile
id: prepare
run: |
echo "hash=$(sha256sum ./dash/contrib/containers/guix/Dockerfile | cut -d ' ' -f1)" >> $GITHUB_OUTPUT
echo "host_user_id=$(id -u)" >> $GITHUB_OUTPUT
echo "host_group_id=$(id -g)" >> $GITHUB_OUTPUT
BRANCH_NAME=$(echo "${GITHUB_REF##*/}" | tr '[:upper:]' '[:lower:]')
REPO_NAME=$(echo "${{ github.repository }}" | tr '[:upper:]' '[:lower:]')
Comment on lines +33 to +34
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I'd add quotes here:

Suggested change
BRANCH_NAME=$(echo "${GITHUB_REF##*/}" | tr '[:upper:]' '[:lower:]')
REPO_NAME=$(echo "${{ github.repository }}" | tr '[:upper:]' '[:lower:]')
BRANCH_NAME="$(echo "${GITHUB_REF##*/}" | tr '[:upper:]' '[:lower:]')"
REPO_NAME="$(echo "${{ github.repository }}" | tr '[:upper:]' '[:lower:]')"

echo "::set-output name=image-tag::${BRANCH_NAME}"
echo "::set-output name=repo-name::${REPO_NAME}"

- name: Login to GitHub Container Registry
uses: docker/login-action@v3
with:
registry: ghcr.io
username: ${{ github.actor }}
password: ${{ secrets.GITHUB_TOKEN }}

- name: Build Docker image
uses: docker/build-push-action@v5
uses: docker/build-push-action@v6
with:
context: ${{ github.workspace }}/dash
build-args: |
USER_ID=${{ steps.dockerfile.outputs.host_user_id }}
GROUP_ID=${{ steps.dockerfile.outputs.host_group_id }}
USER_ID=${{ steps.prepare.outputs.host_user_id }}
GROUP_ID=${{ steps.prepare.outputs.host_group_id }}
build-contexts: |
docker_root=${{ github.workspace }}/dash/contrib/containers/guix
file: ./dash/contrib/containers/guix/Dockerfile
load: true
tags: guix_ubuntu:latest
cache-from: type=gha
cache-to: type=gha,mode=max
push: true
tags: |
ghcr.io/${{ steps.prepare.outputs.repo-name }}/dashcore-guix-builder:${{ steps.prepare.outputs.image-tag }}
ghcr.io/${{ steps.prepare.outputs.repo-name }}/dashcore-guix-builder:latest
cache-from: type=registry,ref=ghcr.io/${{ steps.prepare.outputs.repo-name }}/dashcore-guix-builder:latest
cache-to: type=inline,mode=max

build:
needs: build-image
# runs-on: [ "self-hosted", "linux", "x64", "ubuntu-core" ]
runs-on: ubuntu-latest
# if: ${{ contains(github.event.pull_request.labels.*.name, 'guix-build') }}
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

nit: dead code in comment?

strategy:
matrix:
build_target: [x86_64-linux-gnu, arm-linux-gnueabihf, aarch64-linux-gnu, riscv64-linux-gnu, x86_64-w64-mingw32, x86_64-apple-darwin, arm64-apple-darwin]
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

nit: the list of targets are duplicated now between this yml file and contrib/guix/guix-build:

# Default to building for all supported HOSTs (overridable by environment)
export HOSTS="${HOSTS:-x86_64-linux-gnu arm-linux-gnueabihf aarch64-linux-gnu riscv64-linux-gnu
                       x86_64-w64-mingw32
                       x86_64-apple-darwin arm64-apple-darwin}"

is any easy way to unify it and keep list somewhere such as contrib/guix/util/build_targets ?

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Not that I can think of... no... Open to ideas


- name: Restore Guix cache and depends
timeout-minutes: 480
steps:
- name: Checkout
uses: actions/checkout@v4
with:
ref: ${{ github.event.pull_request.head.sha }}
path: dash
fetch-depth: 0

- name: Cache Guix and depends
id: guix-cache-restore
uses: actions/cache/restore@v3
uses: actions/cache@v3
with:
path: |
${{ github.workspace }}/.cache
${{ github.workspace }}/dash/depends/built
${{ github.workspace }}/dash/depends/sources
${{ github.workspace }}/dash/depends/work
key: ${{ runner.os }}-guix
/gnu/store
key: ${{ runner.os }}-guix-${{ matrix.build_target }}-${{ github.sha }}
restore-keys: |
${{ runner.os }}-guix-${{ matrix.build_target }}
${{ runner.os }}-guix-

- name: Create .cache folder if missing
if: steps.guix-cache-restore.outputs.cache-hit != 'true'
Expand All @@ -67,8 +105,8 @@ jobs:
-v ${{ github.workspace }}/dash:/src/dash \
-v ${{ github.workspace }}/.cache:/home/ubuntu/.cache \
-w /src/dash \
guix_ubuntu:latest && \
docker exec guix-daemon bash -c '/usr/local/bin/guix-start'
ghcr.io/${{ needs.build-image.outputs.repo-name }}/dashcore-guix-builder:${{ needs.build-image.outputs.image-tag }} && \
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

should there be quotes?

Suggested change
ghcr.io/${{ needs.build-image.outputs.repo-name }}/dashcore-guix-builder:${{ needs.build-image.outputs.image-tag }} && \
"ghcr.io/${{ needs.build-image.outputs.repo-name }}/dashcore-guix-builder:${{ needs.build-image.outputs.image-tag }}" && \

docker exec guix-daemon bash -c 'HOSTS=${{ matrix.build_target }} /usr/local/bin/guix-start'

- name: Ensure build passes
run: |
Expand All @@ -77,17 +115,15 @@ jobs:
exit 1
fi

- name: Save Guix cache and depends
id: guix-cache-save
uses: actions/cache/save@v3
- name: Compute SHA256 checksums
continue-on-error: true # It will complain on depending on only some hosts
run: |
HOSTS=${{ matrix.build_target }} ./dash/contrib/containers/guix/scripts/guix-check ${{ github.workspace }}/dash
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

should it be quotes there?

Suggested change
HOSTS=${{ matrix.build_target }} ./dash/contrib/containers/guix/scripts/guix-check ${{ github.workspace }}/dash
HOSTS=${{ matrix.build_target }} ./dash/contrib/containers/guix/scripts/guix-check "${{ github.workspace }}/dash"


- name: Upload build artifacts
uses: actions/upload-artifact@v4
with:
name: guix-artifacts-${{ matrix.build_target }}
path: |
${{ github.workspace }}/.cache
${{ github.workspace }}/dash/depends/built
${{ github.workspace }}/dash/depends/sources
${{ github.workspace }}/dash/depends/work
key: ${{ steps.guix-cache-restore.outputs.cache-primary-key }}
${{ github.workspace }}/dash/guix-build*/output/${{ matrix.build_target }}/
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

should there be quotes?

Suggested change
${{ github.workspace }}/dash/guix-build*/output/${{ matrix.build_target }}/
"${{ github.workspace }}/dash/guix-build*/output/${{ matrix.build_target }}/"

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

umm, I don't think there are needed to be quotes


- name: Compute SHA256 checksums
run: |
./dash/contrib/containers/guix/scripts/guix-check ${{ github.workspace }}/dash
Loading