Skip to content

Commit

Permalink
Rewrite releasing code, add descriptions to releases (#724)
Browse files Browse the repository at this point in the history
  • Loading branch information
Akuli authored Feb 4, 2025
1 parent d995c91 commit f0e0220
Show file tree
Hide file tree
Showing 4 changed files with 153 additions and 84 deletions.
10 changes: 10 additions & 0 deletions .github/workflows/checks.yml
Original file line number Diff line number Diff line change
Expand Up @@ -31,3 +31,13 @@ jobs:
python-version: '3.13'
- run: pip install requests
- run: python check-markdown-links.py --retry=5

# Run the release script to make sure it doesn't error.
release-dry-run:
timeout-minutes: 2
runs-on: ubuntu-latest
steps:
- uses: actions/checkout@v4
with:
fetch-depth: 0 # Fetch the whole Git history, used for release description
- run: ./release.sh --github-token "${{ secrets.GITHUB_TOKEN }}" --dry-run
73 changes: 2 additions & 71 deletions .github/workflows/release.yml
Original file line number Diff line number Diff line change
Expand Up @@ -16,75 +16,6 @@ jobs:
timeout-minutes: 5
steps:
- uses: actions/checkout@v4
- run: git fetch --tags
- id: tagname
run: |
if git tag -l --contains HEAD | grep .; then
# Commit already tagged
echo datetag= >> $GITHUB_OUTPUT
else
echo datetag=$(date +'%Y-%m-%d-%H00') > $GITHUB_OUTPUT
fi
- if: ${{ steps.tagname.outputs.datetag != '' }}
name: Download latest GitHub actions build results
uses: actions/github-script@v6
with:
github-token: ${{secrets.GITHUB_TOKEN}}
script: |
const fs = require('fs');
// Based on: https://github.com/python/typeshed/blob/82fa8473ffddc57a53b4dbcb1063aa2e66352ca9/.github/workflows/mypy_primer_comment.yml
const allRuns = (
await github.rest.actions.listWorkflowRunsForRepo({
owner: context.repo.owner,
repo: context.repo.repo,
branch: 'main',
})
).data.workflow_runs;
console.log(`Found ${allRuns.length} runs`);
console.log(allRuns.map(r => r.name));
const run = allRuns
.filter(r => r.name === '.github/workflows/windows.yml')
.sort((a, b) => (+new Date(b.run_started_at)) - (+new Date(a.run_started_at)))[0];
const allArtifacts = (
await github.rest.actions.listWorkflowRunArtifacts({
owner: context.repo.owner,
repo: context.repo.repo,
run_id: run.id,
})
).data.artifacts;
console.log(`Found ${allArtifacts.length} artifacts`);
console.log(allArtifacts.map(a => a.name));
const artifact = allArtifacts.filter(a => a.name === 'windows-zip')[0];
const zip = await github.rest.actions.downloadArtifact({
owner: context.repo.owner,
repo: context.repo.repo,
artifact_id: artifact.id,
archive_format: 'zip',
});
// https://stackoverflow.com/a/46779188
fs.writeFileSync("nested-zip-file.zip", Buffer.from(zip.data));
# We get a zip file inside a zip file:
# * Inner zip file: The build creates a releasing-ready zip file. This is
# good because you can test the zip file before Jou is released.
# * Outer zip file: It is possible to include multiple files to the same
# GitHub Actions artifact, and downloadArtifact() gives a zip of all
# files that the artifact consists of.
- if: ${{ steps.tagname.outputs.datetag != '' }}
run: unzip nested-zip-file.zip

- if: ${{ steps.tagname.outputs.datetag != '' }}
run: mv jou.zip jou_windows_64bit_${{ steps.tagname.outputs.datetag }}.zip

- if: ${{ steps.tagname.outputs.datetag != '' }}
name: Create release
uses: ncipollo/release-action@v1
with:
commit: main
tag: ${{ steps.tagname.outputs.datetag }}
artifacts: jou_windows_64bit_${{ steps.tagname.outputs.datetag }}.zip
skipIfReleaseExists: true
fetch-depth: 0 # Fetch the whole Git history, used for release description
- run: ./release.sh --github-token "${{ secrets.GITHUB_TOKEN }}"
22 changes: 9 additions & 13 deletions CONTRIBUTING.md
Original file line number Diff line number Diff line change
Expand Up @@ -173,7 +173,7 @@ It only looks at code examples that contain `# Output:`, `# Warning:` or `# Erro
It then attempts to run each example and compares the output similarly to `runtests.sh`.
## Windows Release Builds
## Releases
Using Jou on Linux is easy,
because you can install Jou's dependencies with `apt` (or whatever package manager you have).
Expand All @@ -184,20 +184,16 @@ See the README for instructions to install Jou from a zip file.
A zip file is built and released in GitHub Actions every morning at 4AM UTC,
or when triggered manually from GitHub's UI
(e.g. to release an important bugfix as quickly as possible).
No new release is made if there are no new commits.
Sometimes a pull request doesn't affect Jou users in any way, but makes things easier for Jou developers.
These pull requests should be marked with the `skip-release` label in GitHub.
Now new release is made if there are no new commits or they all have the `skip-release` label.
Some parts of the build are done in `.github/workflows/windows.yml`,
and the rest is in `.github/workflows/release.yml`.
The difference is that `windows.yml` runs on every pull request and on every push to `main`,
but `release.yml` runs daily and when triggered manually.
This means that:
- `windows.yml` should do most of the build.
It should also run tests on the build results to make sure that everything works.
If something in this file stops working, it will be noticed very quickly when someone makes a pull request.
- `release.yml` should be simple and small.
If `release.yml` breaks, it might take a while for someone to notice it.
Ideally it would only download the build results of `windows.yml` and create a release.
and the rest is in `release.sh` (invoked from `.github/workflows/release.yml`).
The difference is that `windows.yml` builds and tests the Windows zip file,
and `release.sh` creates a GitHub release for it.
There is also `jou --update`.
On Linux it simply runs `git pull` and recompiles the Jou compiler.
On Windows it runs a PowerShell script that downloads and installs the latest release from GitHub Actions.
On other platforms it simply runs `git pull` and recompiles the Jou compiler.
132 changes: 132 additions & 0 deletions release.sh
Original file line number Diff line number Diff line change
@@ -0,0 +1,132 @@
#!/bin/bash
#
# This scripts generates a new Jou release. It is invoked from GitHub Actions
# automatically as documented in CONTRIBUTING.md.

set -e -o pipefail

dry_run=no
auth_header=""

while [ $# != 0 ]; do
case "$1" in
--dry-run)
dry_run=yes
shift
;;
--github-token)
auth_header="Authorization: token $2"
shift 2
;;
*)
echo "Usage: $0 [--github-token TOKEN] [--dry-run]"
echo ""
echo "With --dry-run, no release is made. This can be used to develop many"
echo "features of this script."
echo ""
echo "Passing a GitHub API token may be useful even with --dry-run. GitHub API"
echo "is used to check which pull requests have the skip-release label, and you"
echo "may run into rate-limit issues if you don't specify a token."
exit 2
;;
esac
done


echo "Generating release description"
rm -rvf tmp/release
mkdir -vp tmp/release
cd tmp/release

# Make sure git knows about the latest tags in github
git fetch --tags -q

latest_tag=$(git describe --tags --abbrev=0)
for commit in $(git log --pretty=%H $latest_tag..origin/main --reverse); do
# Get first line of commit message (the summary line)
summary="$(git show -s --format=%s $commit)"
echo "Found commit ${commit:0:10}: $summary"

pr_number="$(echo "$summary" | grep -oE '\(#[0-9]+\)$' | grep -oE '[0-9]+')"
if [ -n "$pr_number" ]; then
if curl -s -H "$auth_header" https://api.github.com/repos/Akuli/jou/pulls/$pr_number | jq -r '.labels[].name' | grep -q '^skip-release$'; then
echo " Skipping because pull request #$pr_number has the 'skip-release' label"
continue
fi
link=https://github.com/Akuli/jou/pulls/$pr_number
else
echo " No associated pull request, the commit has probably been pushed directly to main"
link=https://github.com/Akuli/jou/commit/$commit
fi

if ! [ -f desc.md ]; then
echo "This release contains the following changes:" > desc.md
fi
echo "- [$summary]($link)" >> desc.md
done

if ! [ -f desc.md ]; then
echo "No changes to release. Stopping."
exit 0
fi

echo ""
cat desc.md
echo ""

echo "Finding latest Windows CI run for main branch..."
commit=$(git rev-parse origin/main)
curl -s -H "$auth_header" "https://api.github.com/repos/Akuli/jou/actions/runs?branch=main&head_sha=$commit" | jq '.workflow_runs[] | select(.path==".github/workflows/windows.yml")' > run.json
echo " Run ID: $(jq -r .id < run.json)"
echo " Status: $(jq -r .status < run.json)"
echo " Artifacts URL: $(jq -r .artifacts_url < run.json)"
echo ""

if [ $(jq -r .status < run.json) != completed ]; then
echo "Seems like CI is still running. Not releasing."
exit 0
fi

echo "Finding windows-zip artifact..."
archive_download_url="$(curl -s -H "$auth_header" "$(jq -r .artifacts_url < run.json)" | jq -r '.artifacts[] | select(.name == "windows-zip") | .archive_download_url')"
echo " Archive download URL: $archive_download_url"
echo ""

if [ -z "$auth_header" ]; then
echo "Error: GitHub requires authentication to download artifacts. Cannot continue without a token."
exit 1
fi

# We get a zip file inside a zip file:
# * Inner zip file: The build creates a releasing-ready zip file. This is
# good because you can test the zip file before Jou is released.
# * Outer zip file: It is possible to include multiple files to the same
# GitHub Actions artifact, and downloadArtifact() gives a zip of all
# files that the artifact consists of.
echo "Downloading windows-zip artifact..."
curl -L -H "$auth_header" "$archive_download_url" -o nested-zip-file.zip
unzip nested-zip-file.zip
echo ""

datetag=$(date +'%Y-%m-%d-%H00')
jq -n --arg tag $datetag --arg desc "$(cat desc.md)" '{tag_name: $tag, body: $desc}' > release-params.json
echo "Release parameters:"
cat release-params.json
echo ""

if [ $dry_run = yes ]; then
echo "Aborting because this is a dry-run."
exit 0
fi

# To test this, you can change Akuli/jou to a temporary private repo
echo "CREATING RELEASE!!!"
curl -X POST "https://api.github.com/repos/Akuli/jou/releases" -H "$auth_header" -d @release-params.json | tee release-response.json
echo ""

echo "Attaching the Windows zip file to the release..."
# Response contains:
# "upload_url": "https://uploads.github.com/repos/Akuli/jou/releases/xxxxxxxxx/assets{?name,label}",
upload_url="$(jq -r .upload_url < release-response.json | cut -d'{' -f1)?name=jou_windows_64bit_$datetag.zip"
echo " POSTing to $upload_url"
curl -L -X POST -H "$auth_header" -H "Content-Type: application/zip" "$upload_url" --data-binary "@jou.zip"

0 comments on commit f0e0220

Please sign in to comment.