Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Speed up ci:reset by passing along a pre-built DB. #1629

Closed
wants to merge 2 commits into from
Closed

Speed up ci:reset by passing along a pre-built DB. #1629

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
2 commits
Select commit Hold shift + click to select a range
262d9f3
Speed up ci:reset by passing along a pre-built DB.
rasben Oct 3, 2024
5c688e2
Move CI site tests in seperate file, and add re-useable site-reset
rasben Oct 4, 2024
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
24 changes: 24 additions & 0 deletions .github/actions/site-reset/action.yml
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -0,0 +1,24 @@
---
name: Re-setting site, for CI Tests
description: Setup Docker with caching and Buildx

inputs:
token:
description: A GitHub token
required: true

runs:
using: "composite"
steps:
- uses: actions/checkout@v4
with:
# Our Taskfile requires a proper checkout to function because of
# certain vars.
fetch-depth: 0
- name: Install go-task
uses: arduino/setup-task@v2
with:
repo-token: ${{ inputs.token }}
- name: Setup site
shell: bash
run: task ci:reset
98 changes: 98 additions & 0 deletions .github/workflows/ci-site-tests.yml
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -0,0 +1,98 @@
---
name: CI site tests
on: pull_request

jobs:
LightHouse:
name: Test site performance using Lighthouse
runs-on: ubuntu-latest
steps:
- uses: actions/checkout@v4
- name: Re-setting site
uses: ./.github/actions/site-reset
with:
token: ${{ secrets.GITHUB_TOKEN }}
- name: Run Lighthouse test
run: task ci:lighthouse
- name: Archive logs
uses: actions/upload-artifact@v4
if: ${{ always() }}
with:
name: lighthouse-results
path: .lighthouseci

Pa11y:
name: Test accessibility using Pa11y
runs-on: ubuntu-latest
steps:
- uses: actions/checkout@v4
- name: Re-setting site
uses: ./.github/actions/site-reset
with:
token: ${{ secrets.GITHUB_TOKEN }}
- name: Run Pa11y
run: task ci:pa11y
- name: Archive screenshots
if: ${{ always() }}
uses: actions/upload-artifact@v4
with:
name: pa11y-screenshots
path: pa11y/screenshots

Cypress:
name: Run Cypress functional tests
runs-on: ubuntu-latest
steps:
- uses: actions/checkout@v4
- name: Re-setting site
uses: ./.github/actions/site-reset
with:
token: ${{ secrets.GITHUB_TOKEN }}
- name: Run Cypress
run: task ci:cypress
- name: Archive videoes
if: ${{ always() }}
uses: actions/upload-artifact@v4
with:
name: cypress-videos
path: cypress/videos
- name: Archive screenshots
if: ${{ always() }}
uses: actions/upload-artifact@v4
with:
name: cypress-screenshots
path: cypress/screenshots

CheckOpenApiSpec:
name: Check OpenAPI specification
runs-on: ubuntu-latest
steps:
- uses: actions/checkout@v4
- name: Re-setting site
uses: ./.github/actions/site-reset
with:
token: ${{ secrets.GITHUB_TOKEN }}
- name: Validate specification
run: task ci:openapi:validate
- name: Download current specification
run: task ci:openapi:download
- name: Ensure specification has not drifted
run: git diff --ignore-space-at-eol --exit-code openapi.json
- name: Generate package for CMS API specification
run: task dev:codegen:dpl-cms
- name: Ensure CMS API package code has not drifted
run: git diff --ignore-space-at-eol --exit-code packages/cms-api/*

CheckDrupalConfig:
name: Check Drupal Config
runs-on: ubuntu-latest
steps:
- uses: actions/checkout@v4
- name: Re-setting site
uses: ./.github/actions/site-reset
with:
token: ${{ secrets.GITHUB_TOKEN }}
- name: Export configuration
run: task dev:cli -- drush config-export -y
- name: Check for uncommited configuration after install
run: git diff --ignore-space-at-eol --exit-code config/sync/*.yml
123 changes: 0 additions & 123 deletions .github/workflows/ci-tests.yml
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -102,83 +102,6 @@ jobs:
- name: Run PhpUnit
run: ./vendor/bin/phpunit --coverage-text

LightHouse:
name: Test site performance using Lighthouse
runs-on: ubuntu-latest
steps:
- uses: actions/checkout@v4
with:
# Our Taskfile requires a proper checkout to function because of
# certain vars.
fetch-depth: 0
- name: Install go-task
uses: arduino/setup-task@v2
with:
repo-token: ${{ secrets.GITHUB_TOKEN }}
- name: Setup site
run: task ci:reset
- name: Run Lighthouse test
run: task ci:lighthouse
- name: Archive logs
uses: actions/upload-artifact@v4
with:
name: lighthouse-results
path: .lighthouseci

Pa11y:
name: Test accessibility using Pa11y
runs-on: ubuntu-latest
steps:
- uses: actions/checkout@v4
with:
# Our Taskfile requires a proper checkout to function because of
# certain vars.
fetch-depth: 0
- name: Install go-task
uses: arduino/setup-task@v2
with:
repo-token: ${{ secrets.GITHUB_TOKEN }}
- name: Setup site
run: task ci:reset
- name: Run Pa11y
run: task ci:pa11y
- name: Archive screenshots
if: ${{ always() }}
uses: actions/upload-artifact@v4
with:
name: pa11y-screenshots
path: pa11y/screenshots

Cypress:
name: Run Cypress functional tests
runs-on: ubuntu-latest
steps:
- uses: actions/checkout@v4
with:
# Our Taskfile requires a proper checkout to function because of
# certain vars.
fetch-depth: 0
- name: Install go-task
uses: arduino/setup-task@v2
with:
repo-token: ${{ secrets.GITHUB_TOKEN }}
- name: Setup site
run: task ci:reset
- name: Run Cypress
run: task ci:cypress
- name: Archive videoes
if: always()
uses: actions/upload-artifact@v4
with:
name: cypress-videos
path: cypress/videos
- name: Archive screenshots
if: always()
uses: actions/upload-artifact@v4
with:
name: cypress-screenshots
path: cypress/screenshots

MarkdownLint:
name: Lint Markdown
runs-on: ubuntu-latest
Expand Down Expand Up @@ -256,52 +179,6 @@ jobs:
- name: Run ESLint
run: npx eslint cypress -c cypress/.eslintrc.json

CheckOpenApiSpec:
name: Check OpenAPI specification
runs-on: ubuntu-latest
steps:
- uses: actions/checkout@v4
with:
# Our Taskfile requires a proper checkout to function because of
# certain vars.
fetch-depth: 0
- name: Install go-task
uses: arduino/setup-task@v2
with:
repo-token: ${{ secrets.GITHUB_TOKEN }}
- name: Setup site
run: task ci:reset
- name: Validate specification
run: task ci:openapi:validate
- name: Download current specification
run: task ci:openapi:download
- name: Ensure specification has not drifted
run: git diff --ignore-space-at-eol --exit-code openapi.json
- name: Generate package for CMS API specification
run: task dev:codegen:dpl-cms
- name: Ensure CMS API package code has not drifted
run: git diff --ignore-space-at-eol --exit-code packages/cms-api/*

CheckDrupalConfig:
name: Check Drupal Config
runs-on: ubuntu-latest
steps:
- uses: actions/checkout@v4
with:
# Our Taskfile requires a proper checkout to function because of
# certain vars.
fetch-depth: 0
- name: Install go-task
uses: arduino/setup-task@v2
with:
repo-token: ${{ secrets.GITHUB_TOKEN }}
- name: Setup site
run: task ci:reset
- name: Export configuration
run: task dev:cli -- drush config-export -y
- name: Check for uncommited configuration after install
run: git diff --ignore-space-at-eol --exit-code config/sync/*.yml

CheckPhpVersion:
name: Check PHP version
runs-on: ubuntu-latest
Expand Down
34 changes: 20 additions & 14 deletions Taskfile.yml
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -125,8 +125,8 @@ tasks:
cmds:
- docker compose {{ .DOCKER_COMPOSE_FILES }} pull

dev:reset:
desc: Create local development setup in a clean state
dev:reset:common:
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

What is the purpose of dev:reset:common? Would be nice to have in the description what the intention is.
Also consider which of the newly introduced dev:rest* cmds private so it is clear what is meant to be run as seprate task cmds and what is used internally

desc: Preparing the environments for resetting the setup.
deps:
# Build new containers if necessary.
- dev:build
Expand All @@ -149,22 +149,16 @@ tasks:
- task dev:pull
# Start local environment.
- task dev:start

dev:reset:
desc: Create local development setup in a clean state
cmds:
- task dev:reset:before
# Install site.
- task dev:cli -- drush site-install --existing-config -y
# Practice shows that the cache needs to be cleared to avoid configuration
# errors even after a site install.
- task dev:cache:clear:drupal
# Import translations.
- |
if [ -n "${SKIP_LANGUAGE_IMPORT}" ]; then
echo "Skipping language import due to SKIP_LANGUAGE_IMPORT environment variable"
else
task dev:cli -- drush locale-check
task dev:cli -- drush locale-update
task dev:cli -- drush dpl_po:import-remote-config-po da https://danskernesdigitalebibliotek.github.io/dpl-cms/translations/da.config.po
fi
# Clear all caches to ensure we have a pristine setup.
- task dev:cache:clear:all
# Run deploy hooks.
- task dev:cli -- drush deploy -y
# Ensure site is reachable and warm any caches
Expand All @@ -174,6 +168,16 @@ tasks:
- task dev:create-users
# Show a one-time login to the local site.
- task dev:cli -- drush user-login
# Import translations.
- |
if [ -n "${SKIP_LANGUAGE_IMPORT}" ]; then
echo "Skipping language import due to SKIP_LANGUAGE_IMPORT environment variable"
else
task dev:cli -- drush locale-check
task dev:cli -- drush locale-update
task dev:cli -- drush dpl_po:import-remote-config-po da https://danskernesdigitalebibliotek.github.io/dpl-cms/translations/da.config.po
task dev:import-profile-translations
fi

dev:openid:configure:
desc: Set openid connect settings based on .env variables. And run cron.
Expand Down Expand Up @@ -382,7 +386,9 @@ tasks:
ci:reset:
desc: Create CI setup in a clean state
cmds:
- task dev:reset
- task dev:reset:common
- task dev:cli -- drush sqlq --file=../tools/site-reset.sql.gz
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

What is contained in site-reset.sql.gz?
It would be nice if it was documented somewhere what is happening here

- task dev:cli -- drush deploy
env:
DOCKER_COMPOSE_FILES: "{{ .DOCKER_COMPOSE_FILES_CI }}"
SKIP_LANGUAGE_IMPORT: "true"
Expand Down
Binary file added tools/site-reset.sql.gz
View file
Open in desktop
Binary file not shown.
Loading