Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

infra: automatically post documentation previews for PRs #4829

Closed
wants to merge 3 commits into from
Closed

infra: automatically post documentation previews for PRs #4829

Show file tree
Hide file tree
Changes from 1 commit
Commits
Show all changes
3 commits
Select commit Hold shift + click to select a range
b292402
docs: migrate from poetry to uv
neongreen Nov 9, 2024
1039b97
remove all other workflows (to be undone later)
neongreen Nov 10, 2024
6de1abd
infra: publish PR previews for docs
neongreen Nov 10, 2024
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
49 changes: 49 additions & 0 deletions .github/workflows/docs-preview-build.yml
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -0,0 +1,49 @@

# This workflow builds the docs for each PR (including from forks), and uploads
# the rendered docs as an artifact.
#
# The artifact will be picked up by docs-preview-deploy.yml and uploaded to
# https://jj-preview.github.io/docs/.
#
# The split into "build" and "deploy" follows GitHub's guide at
# https://securitylab.github.com/resources/github-actions-preventing-pwn-requests/.
# The build workflow has no access to the repo secrets. The deploy workflow runs
# in the context of the main repo rather than the PR branch.
neongreen marked this conversation as resolved.
Show resolved Hide resolved

name: docs-preview-build

on:
pull_request:
ilyagr marked this conversation as resolved.
Show resolved Hide resolved
paths:
- 'docs/**'
- 'mkdocs.yml'
- 'pyproject.toml'
- 'uv.lock'
- '.github/workflows/docs-preview-build.yml'

jobs:
build:
runs-on: ubuntu-latest
steps:
- uses: actions/checkout@11bd71901bbe5b1630ceea73d27597364c9af683

- name: Install uv
uses: astral-sh/setup-uv@2e657c127d5b1635d5a8e3fa40e0ac50a5bf6992

- name: Build the docs
# Intentionally without --strict, to have previews even if the docs are
# mildly broken
run: uv run mkdocs build

- uses: actions/upload-artifact@b4b15b8c7c6ac21ea08fcf65892d2ee8f75cf882
with:
name: rendered-docs
path: rendered-docs

- run: |
echo ${{ github.event.pull_request.number }} > pr_number

- uses: actions/upload-artifact@b4b15b8c7c6ac21ea08fcf65892d2ee8f75cf882
with:
name: pr_number
path: pr_number
32 changes: 32 additions & 0 deletions .github/workflows/docs-preview-cleanup.yml
View file
Open in desktop
ilyagr marked this conversation as resolved.
Show resolved Hide resolved
Original file line number Diff line number Diff line change
@@ -0,0 +1,32 @@
name: docs-preview-cleanup

permissions:
contents: write

on:
# This always uses the workflow file from the base branch, and only runs if
# base == main
pull_request_target:
types: [closed]

jobs:
cleanup:
runs-on: ubuntu-latest
steps:
- uses: actions/checkout@11bd71901bbe5b1630ceea73d27597364c9af683

- run: mkdir -p empty-dir

- name: Deploy
uses: JamesIves/github-pages-deploy-action@62fec3add6773ec5dbbf18d2ee4260911aa35cf4
with:
folder: empty-dir
Copy link
Contributor

@ilyagr ilyagr Nov 17, 2024 •
edited
Loading

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

(Thinking out loud, we can fiddle with this later if necessary) This might create a lot of empty dirs.

I don't think it matters much, but perhaps a better approach would be:

  • Deploy a file called PR_CLOSED to the dir in the docs repo here. No need to delete anything.
  • Have an action in the docs repo that deletes all dirs with PR_CLOSED marker if it's over 7 days old.

I'm not sure how well that will work if that action happens to happen while another action is deploying to the docs. But I'm also not sure it'd be worse than with this PR as is; I'm not sure how this PR will do if the cleanup action on one PR is running at the same time as the deploy action acts on another PR.

It is something we can discover experimentally, though.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This might create a lot of empty dirs.

It actually deletes the dir entirely (I checked).

I'm not sure how well that will work if that action happens to happen while another action is deploying to the docs.

Good catch. In fact, GHA doesn't support a "workflow queue" at all — if you use concurrency:, it will only keep one pending job in the queue.

I will fix that by disabling force and single-commit and switching to per-PR concurrency groups.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Update: I will also delete the cleanup entirely b/c the extra complexity is not worth it

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

(Thinking out loud) Would just adding the PR_CLOSED file (or something like that) be too complex as well? Then we could occasionally clean up manually.

Unless we nuke the repo history, garbage collection won't save much git storage, but it might save on the amount of traffic needed to deploy the website. OTOH, GitHub seems to take care of that, so maybe it's not a problem (or we could assume that they are smart about it and use rsync).

Ultimately, this is up to you.

repository-name: jj-preview/docs
branch: 'main'
clean: true
target-folder: 'pr-${{ github.event.pull_request.number }}'
ssh-key: ${{ secrets.DOCS_DEPLOY_KEY }}
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Might make sense to add a comment: # Creating this is documented in docs-preview-deploy.yml.

git-config-name: 'jj-docs[bot]'
git-config-email: 'jj-docs[bot]@users.noreply.github.io'
commit-message: "Remove preview for PR ${{ github.event.pull_request.number }}"
single-commit: true
93 changes: 93 additions & 0 deletions .github/workflows/docs-preview-deploy.yml
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -0,0 +1,93 @@
# This workflow pushes docs rendered by docs-preview-build.yml to
# https://github.com/jj-preview/docs.
#
# There, GitHub's own "deploy pages" action will run to publish the static files
# to https://jj-preview.github.io/docs/. This adds a 30s delay before the docs
# are actually published/updated.
#
# jj-preview/docs has a top-level .nojekyll file. Otherwise GitHub will try
# rendering with Jekyll, and the delay will be longer.
#
# We use a separate repo for two reasons:
# 1. No chance to accidentally mess up the main GH Pages
# 2. Avoid the impression that the published content is "official"

# TODO: PRs touching workflows should be approved by a maintainer
# TODO: enable docs-preview-build for PRs from forks

name: docs-preview-deploy

permissions:
contents: write
pull-requests: write

on:
# NB: 'workflow_run' only triggers if this workflow is on the default branch,
# which is needed for security but makes it a bit annoying to test
workflow_run:
workflows: [docs-preview-build]
types: [completed]

# Limit to one concurrent deployment.
#
# If we ever need parallel deployments, we'll need single-commit=false and
# force=false in the deploy step.
concurrency:
group: preview_deploy

jobs:
deploy:
runs-on: ubuntu-latest
if: ${{ github.event.workflow_run.conclusion == 'success' }}
steps:
- uses: actions/checkout@11bd71901bbe5b1630ceea73d27597364c9af683

- uses: actions/download-artifact@fa0a91b85d4f404e444e00e005971372dc801d16
with:
name: rendered-docs
path: rendered-docs
github-token: ${{ secrets.GITHUB_TOKEN }}
run-id: ${{ github.event.workflow_run.id }}

- uses: actions/download-artifact@fa0a91b85d4f404e444e00e005971372dc801d16
with:
name: pr_number
github-token: ${{ secrets.GITHUB_TOKEN }}
run-id: ${{ github.event.workflow_run.id }}

# Security note: The PR number is uploaded by the build workflow, and can
# be manipulated. The only thing an attacker can do, though, is overwrite
# someone else's docs preview. See
# https://github.com/orgs/community/discussions/25220
- run: |
echo "PR_NUMBER=$(grep -o '[0-9]\+' pr_number)" >> $GITHUB_ENV

- name: Deploy
uses: JamesIves/github-pages-deploy-action@62fec3add6773ec5dbbf18d2ee4260911aa35cf4
with:
folder: rendered-docs
repository-name: jj-preview/docs
branch: main
clean: true
target-folder: 'pr-${{ env.PR_NUMBER }}'
# This key allows pushing to jj-preview/docs. How to set it up:
#
# 1. ssh-keygen -t ed25519 -C "" -N "" -f <where to put the keypair>
# 2. Add the public key to https://github.com/jj-preview/docs/settings/keys/new
# 3. Add the private key to https://github.com/martinvonz/jj/settings/secrets/actions/new
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I'm guessing only @martinvonz can do this, right?

Speaking of which, I just invited Martin to the jj-preview org. It's a party now. :)

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Yeah, Martin needs to add a GH secret. Or give someone a "CI/CD admin" role:

https://github.blog/changelog/2024-09-25-introducing-ci-cd-admin-a-new-pre-defined-organization-role-for-github-actions/

With the new CI/CD Admin role, organization owners and teams can now delegate comprehensive CI/CD management to individuals without the need to maintain a custom role

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I've added a secret called DOCS_DEPLOY_KEY. Its public key is:

ssh-ed25519 AAAAC3NzaC1lZDI1NTE5AAAAIH5kNau/89p02CNJy0I+ijuE0qZUp+HjhZgZLtJ64m9y

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thank you! I added the public key to https://github.com/jj-preview/docs/settings/keys. I also invited Martin to that org, but there's probably now no immediate need for him to join.

ssh-key: ${{ secrets.DOCS_DEPLOY_KEY }}
git-config-name: 'jj-docs[bot]'
git-config-email: 'jj-docs[bot]@users.noreply.github.io'
commit-message: "Update preview for PR ${{ env.PR_NUMBER }}"
single-commit: true

- name: Post a link to the docs in the PR
uses: thollander/actions-comment-pull-request@65f9e5c9a1f2cd378bd74b2e057c9736982a8e74
with:
message: |
**📖 Documentation preview is published to https://jj-preview.github.io/docs/pr-${{ env.PR_NUMBER }}.**
Thanks for working on the docs!

Copy link
Contributor

@ilyagr ilyagr Nov 16, 2024 •
edited
Loading

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I'd add:

This feature is experimental, feel free to open a bug or tell us in a chat if something is not working. We may remove it if it's too flaky.

You can always build and serve the docs on your own machine by following the instructions at https://martinvonz.github.io/jj/prerelease/contributing/#previewing-the-html-documentation

Note: it will take about 30 seconds before the preview is available.
pr-number: ${{ env.PR_NUMBER }}
comment-tag: docs-preview