Skip to content

Commit

Permalink
Limit docs publishing tests to canary run (apache#37196)
Browse files Browse the repository at this point in the history
Docs publishing step takes a lot of disk space and time to complete, due
to size of the airflow-site repo. We should limit it to only run for
canary runs on self-hosted runners.
  • Loading branch information
potiuk authored Feb 6, 2024
1 parent af30e1b commit b37e792
Show file tree
Hide file tree
Showing 2 changed files with 10 additions and 7 deletions.
12 changes: 6 additions & 6 deletions .github/workflows/ci.yml
Original file line number Diff line number Diff line change
Expand Up @@ -767,25 +767,25 @@ jobs:
run: >
git clone https://github.com/apache/airflow-site.git ${GITHUB_WORKSPACE}/airflow-site &&
echo "AIRFLOW_SITE_DIRECTORY=${GITHUB_WORKSPACE}/airflow-site" >> "$GITHUB_ENV"
if: needs.build-info.outputs.full-tests-needed == 'true'
if: needs.build-info.outputs.canary-run == 'true'
- name: "Publish docs"
run: >
breeze release-management publish-docs
--override-versioned --run-in-parallel
${{ needs.build-info.outputs.docs-list-as-string }}
if: needs.build-info.outputs.full-tests-needed == 'true'
if: needs.build-info.outputs.canary-run == 'true'
- name: "Generate back references for providers"
run: breeze release-management add-back-references all-providers
if: needs.build-info.outputs.full-tests-needed == 'true'
if: needs.build-info.outputs.canary-run == 'true'
- name: "Generate back references for apache-airflow"
run: breeze release-management add-back-references apache-airflow
if: needs.build-info.outputs.full-tests-needed == 'true'
if: needs.build-info.outputs.canary-run == 'true'
- name: "Generate back references for docker-stack"
run: breeze release-management add-back-references docker-stack
if: needs.build-info.outputs.full-tests-needed == 'true'
if: needs.build-info.outputs.canary-run == 'true'
- name: "Generate back references for helm-chart"
run: breeze release-management add-back-references helm-chart
if: needs.build-info.outputs.full-tests-needed == 'true'
if: needs.build-info.outputs.canary-run == 'true'
- name: Configure AWS credentials
uses: aws-actions/configure-aws-credentials@010d0da01d0b5a38af31e9c3470dbfdabdecca3a # v4.0.1
if: needs.build-info.outputs.canary-run == 'true' && needs.build-info.outputs.default-branch == 'main'
Expand Down
5 changes: 4 additions & 1 deletion dev/breeze/doc/ci/05_workflows.md
Original file line number Diff line number Diff line change
Expand Up @@ -237,7 +237,7 @@ code.
| Upgrade checks | Performs checks if there are some pending upgrades | | Yes | Yes | Yes |
| Static checks | Performs full static checks | Yes (6) | Yes | Yes | Yes (7) |
| Basic static checks | Performs basic static checks (no image) | Yes (6) | | | |
| Build docs | Builds and tests publishing of the documentation | Yes | Yes | Yes | Yes |
| Build docs | Builds and tests publishing of the documentation | Yes | Yes (11) | Yes | Yes |
| Spellcheck docs | Spellcheck docs | Yes | Yes | Yes | Yes |
| Tests wheel provider packages | Tests if provider packages can be built and released | Yes | Yes | Yes | |
| Tests Airflow compatibility | Compatibility of provider packages with older Airflow | Yes | Yes | Yes | |
Expand Down Expand Up @@ -293,6 +293,9 @@ via `skip-provider-tests` selective check output.
`(10)` Only run the builds in case PR is run by a committer from
"apache" repository and in scheduled build.

`(11)` Docs publishing is only done in Canary run, to handle the case where
cloning whole airflow site on Public Runner cannot complete due to the size of the repository.

## CodeQL scan

The [CodeQL](https://securitylab.github.com/tools/codeql) security scan
Expand Down

0 comments on commit b37e792

Please sign in to comment.