Skip to content

Commit

Permalink
ci: Refine docs deployment and PR check CI (#4148)
Browse files Browse the repository at this point in the history
<!--
Thanks for your contribution! please review
https://github.com/alibaba/GraphScope/blob/main/CONTRIBUTING.md before
opening an issue.
-->

## What do these changes do?

<!-- Please give a short brief about these changes. -->

- refine PR-check CI
    - do conventional check
    - code lint
- if docs changed, gen and comment a preview URL (deprecated surge and
move to cloudflare )
- refine docs CI, only triggered when a PR is merged and on the main.
  • Loading branch information
yecol authored Aug 16, 2024
1 parent 049fe92 commit 568215b
Show file tree
Hide file tree
Showing 3 changed files with 233 additions and 259 deletions.
191 changes: 3 additions & 188 deletions .github/workflows/docs.yml
Original file line number Diff line number Diff line change
@@ -1,155 +1,20 @@
name: Docs
name: Docs Deployment

on:
workflow_dispatch:
push:
branches:
- main
- docs
- dev/docs
tags:
- "v*"
pull_request:
branches:
- main
- docs
- dev/docs

jobs:
build:
runs-on: ubuntu-20.04
permissions:
issues: write
pull-requests: write
contents: write
steps:
- name: Checkout Code
uses: actions/checkout@v4
with:
repository: ${{ github.event.pull_request.head.repo.full_name }}
ref: ${{ github.event.pull_request.head.ref }}
submodules: true
fetch-depth: 0

- uses: dorny/paths-filter@v2
id: changes
with:
filters: |
src:
- 'docs/**'
- name: Setup Java11
uses: actions/setup-java@v3
with:
distribution: 'zulu'
java-version: '11'

- uses: actions/setup-node@v3
with:
node-version: 16

- name: Leave a marker
if: ${{ steps.changes.outputs.src == 'true' && github.event_name == 'pull_request' }}
run: |
touch ${GITHUB_WORKSPACE}/preview-the-docs.mark
- name: Leave the comment on pull request when started
if: ${{ steps.changes.outputs.src == 'true' && github.event_name == 'pull_request' && github.event.pull_request.head.repo.full_name == 'alibaba/GraphScope' }}
uses: actions-cool/maintain-one-comment@v3
with:
token: ${{ secrets.GITHUB_TOKEN }}
body: |
⚡️ Deploying PR Preview ${{ github.event.pull_request.head.sha }} to [surge.sh](https://alibaba-graphscope-build-pr-${{ github.event.number }}.surge.sh) ... [Build logs](https://github.com/alibaba/GraphScope/runs/${{ github.run_id }})
<a href="https://github.com/alibaba/GraphScope/actions/runs/${{ github.run_id }}">
<img width="300" src="https://user-images.githubusercontent.com/507615/90240294-8d2abd00-de5b-11ea-8140-4840a0b2d571.gif">
</a>
<sub>🤖 By [surge-preview](https://surge.sh/)</sub>
body-include: '<!-- Sticky Pull Request Comment: Surge Preview build -->'

- name: Cpp Format and Lint Check
run: |
# install clang-format
sudo curl -L https://github.com/muttleyxd/clang-tools-static-binaries/releases/download/master-22538c65/clang-format-8_linux-amd64 --output /usr/bin/clang-format
sudo chmod +x /usr/bin/clang-format
# run format
cd analytical_engine/
find ./apps ./benchmarks ./core ./frame ./misc ./test -name "*.h" | xargs clang-format -i --style=file
find ./apps ./benchmarks ./core ./frame ./misc ./test -name "*.cc" | xargs clang-format -i --style=file
# validate format
function prepend() { while read line; do echo "${1}${line}"; done; }
GIT_DIFF=$(git diff --ignore-submodules)
if [[ -n $GIT_DIFF ]]; then
echo "~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~"
echo "| clang-format failures found!"
echo "|"
echo "$GIT_DIFF" | prepend "| "
echo "|"
echo "| Run: "
echo "|"
echo "| make gsa_clformat"
echo "|"
echo "| to fix this error."
echo "|"
echo "| Ensure you are working with clang-format-8, which can be obtained from"
echo "|"
echo "| https://github.com/muttleyxd/clang-tools-static-binaries/releases"
echo "|"
echo "~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~"
exit -1
fi
- name: Java Format and Lint Check
run: |
wget https://github.com/google/google-java-format/releases/download/v1.13.0/google-java-format-1.13.0-all-deps.jar
files_to_format=$(git ls-files *.java)
# run formatter in-place
java -jar ${GITHUB_WORKSPACE}/google-java-format-1.13.0-all-deps.jar --aosp --skip-javadoc-formatting -i $files_to_format
# validate format
function prepend() { while read line; do echo "${1}${line}"; done; }
GIT_DIFF=$(git diff --ignore-submodules)
if [[ -n $GIT_DIFF ]]; then
echo "~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~"
echo "| google-java-format failures found!"
echo "|"
echo "$GIT_DIFF" | prepend "| "
echo "|"
echo "| Run: "
echo "|"
echo '| java -jar google-java-format-1.13.0-all-deps.jar --aosp --skip-javadoc-formatting -i $(git ls-files **/*.java)'
echo "|"
echo "| to fix this error."
echo "|"
echo "| Ensure you are working with google-java-format-1.13.0, which can be obtained from"
echo "|"
echo "| https://github.com/google/google-java-format/releases/download/v1.13.0/google-java-format-1.13.0-all-deps.jar"
echo "|"
echo "~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~"
exit -1
fi
- name: Python Format and Lint Check
run: |
echo "Checking formatting for $GITHUB_REPOSITORY"
pip3 install -r coordinator/requirements-dev.txt
pip3 install -r python/requirements-dev.txt
pushd python
python3 -m isort --check --diff .
python3 -m black --check --diff .
python3 -m flake8 .
popd
pushd coordinator
python3 -m isort --check --diff .
python3 -m black --check --diff .
python3 -m flake8 .
fetch-depth: 1

- name: Generate Docs
shell: bash
Expand All @@ -170,61 +35,11 @@ jobs:
make graphscope-docs
fi
- name: Preview using surge
if: ${{ steps.changes.outputs.src == 'true' && github.event_name == 'pull_request' && github.event.pull_request.head.repo.full_name == 'alibaba/GraphScope' }}
run: |
npm install -g surge
surge ./docs/_build/latest/html \
alibaba-graphscope-build-pr-${{ github.event.number }}.surge.sh \
--token ${{ secrets.SURGE_TOKEN }}
- name: Leave the comment on pull request when succeed
if: ${{ steps.changes.outputs.src == 'true' && github.event_name == 'pull_request' && github.event.pull_request.head.repo.full_name == 'alibaba/GraphScope' }}
uses: actions-cool/maintain-one-comment@v3
with:
token: ${{ secrets.GITHUB_TOKEN }}
body: |
🎊 PR Preview ${{ github.event.pull_request.head.sha }} has been successfully built and deployed to https://alibaba-graphscope-build-pr-${{ github.event.number }}.surge.sh
<a href="https://github.com/alibaba/GraphScope/actions/runs/${{ github.run_id }}">
<img width="300" src="https://user-images.githubusercontent.com/507615/90250366-88233900-de6e-11ea-95a5-84f0762ffd39.png">
</a>
<sub>🤖 By [surge-preview](https://surge.sh/)</sub>
body-include: '<!-- Sticky Pull Request Comment: Surge Preview build -->'

- name: Check file existence
id: check_files
if: ${{ failure() }}
uses: andstor/file-existence-action@v2
with:
files: "preview-the-docs.mark"

- name: Leave the comment on pull request when failed
if: ${{ failure() && steps.check_files.outputs.files_exists == 'true' && github.event.pull_request.head.repo.full_name == 'alibaba/GraphScope' }}
uses: actions-cool/maintain-one-comment@v3
with:
token: ${{ secrets.GITHUB_TOKEN }}
body: |
😭 Deploy PR Preview ${{ github.event.pull_request.head.sha }} failed. [Build logs](https://github.com/alibaba/GraphScope/runs/${{ github.run_id }})
<a href="https://github.com/alibaba/GraphScope/actions/runs/${{ github.run_id }}">
<img width="300" src="https://user-images.githubusercontent.com/507615/90250824-4e066700-de6f-11ea-8230-600ecc3d6a6b.png">
</a>
<sub>🤖 By [surge-preview](https://surge.sh/)</sub>
body-include: '<!-- Sticky Pull Request Comment: Surge Preview build -->'

- name: Setup tmate session
uses: mxschmitt/action-tmate@v3
if: false

- name: Upload Docs
if: ${{ github.event_name == 'push' && github.repository == 'alibaba/GraphScope' && (github.ref == 'refs/heads/main' || startsWith(github.ref, 'refs/tags/v')) }}
if: ${{ github.repository == 'alibaba/GraphScope' }}
shell: bash
run: |
shopt -s extglob
rm google-java-format-1.13.0-all-deps.jar* || true
git config user.name "github-actions[bot]"
git config user.email "41898282+github-actions[bot]@users.noreply.github.com"
Expand Down
Loading

0 comments on commit 568215b

Please sign in to comment.