Skip to content

Commit

Permalink
Auto merge of #5088 - rust-lang:gha, r=<try>
Browse files Browse the repository at this point in the history
[WIP][DNM] Switch to GitHub Actions - Part 2 - From within

This is a continuation of #5071. This time from a branch inside the rust-lang/rust-clippy repo, not from my fork, since secrets are not available in PRs from forks.

Copying the description of #5071 to here:

Closes #4577

~~This is just an experiment. I don't think we have a consensus _if_ we should move away from travis/appveyor.~~ We have consensus: #5071 (comment)

~~GHA would let us run up to 20 concurrent jobs. Since we have 15 integration tests and 4 (linux, linux 32-bit, macos, windows) basic tests, we would be able to run everything concurrently.~~ The org has a limit of 60 jobs across the org, so we limit the matrix of the integration tests to 6 concurrent jobs.

~~Also IIUC we only have to build Clippy once for every initegration test and then only check the repos.~~ Nope, dependent jobs exist, but they won't keep the artifacts (not even the checkout).

TODO before merge:

- [x] Add `DEPLOY_KEY` secret to github repo
- [x] test deployment on test branch `gh-test`#
  - [x] Test normal deployment
  - [x] Test deployment no changes
  - [x] Test deployment of tag
- [x] talk with `@rust-lang/infra` for bors, `@rust-lang/infra` is good with the move (crater also uses GHA+bors)
- [x] ~~Get remark + clippy_dev check to work on pushes (https://github.community/t5/GitHub-Actions/~Builds-are-not-triggered-with-on-paths/m-p/44075; I contacted GH support about this)~~ That seems to start working again yesterday 🤔 Let's hope it keeps working.
- [ ] impl homu checks for GHA #5071 (comment)
- [ ] Add GHA badge to Cargo.toml (blocked on `rust-lang/crates.io#1838`)
- [ ] Add back travis + appveyor files for transition period (?)
  • Loading branch information
bors committed Jan 30, 2020
2 parents f69835b + df9be48 commit 62f1444
Show file tree
Hide file tree
Showing 26 changed files with 692 additions and 460 deletions.
60 changes: 11 additions & 49 deletions .github/deploy.sh
100755 → 100644
Original file line number Diff line number Diff line change
@@ -1,20 +1,7 @@
#!/bin/bash

# Automatically deploy on gh-pages
#! /bin/bash

set -ex

SOURCE_BRANCH="master"
TARGET_BRANCH="gh-pages"

# Save some useful information
REPO=$(git config remote.origin.url)
SSH_REPO=${REPO/https:\/\/github.com\//git@github.com:}
SHA=$(git rev-parse --verify HEAD)

# Clone the existing gh-pages for this repo into out/
git clone --quiet --single-branch --branch "$TARGET_BRANCH" "$REPO" out

echo "Removing the current docs for master"
rm -rf out/master/ || exit 0

Expand All @@ -23,59 +10,34 @@ mkdir out/master/
cp util/gh-pages/index.html out/master
python ./util/export.py out/master/lints.json

if [[ -n "$TRAVIS_TAG" ]]; then
echo "Save the doc for the current tag ($TRAVIS_TAG) and point current/ to it"
cp -r out/master "out/$TRAVIS_TAG"
rm -f out/current
ln -s "$TRAVIS_TAG" out/current
if [[ -n $TAG_NAME ]]; then
echo "Save the doc for the current tag ($TAG_NAME) and point current/ to it"
cp -r out/master "out/$TAG_NAME"
rm -f out/current
ln -s "$TAG_NAME" out/current
fi

# Generate version index that is shown as root index page
cp util/gh-pages/versions.html out/index.html
pushd out

cd out
cat <<-EOF | python - > versions.json
import os, json
print json.dumps([
dir for dir in os.listdir(".") if not dir.startswith(".") and os.path.isdir(dir)
])
EOF
popd

# Pull requests and commits to other branches shouldn't try to deploy, just build to verify
if [[ "$TRAVIS_PULL_REQUEST" != "false" ]] || [[ "$TRAVIS_BRANCH" != "$SOURCE_BRANCH" ]]; then
# Tags should deploy
if [[ -z "$TRAVIS_TAG" ]]; then
echo "Generated, won't push"
exit 0
fi
fi

# Now let's go have some fun with the cloned repo
cd out
git config user.name "Travis CI"
git config user.email "[email protected]"
git config user.name "GHA CI"
git config user.email "[email protected]"

if git diff --exit-code --quiet; then
echo "No changes to the output on this push; exiting."
exit 0
echo "No changes to the output on this push; exiting."
exit 0
fi
cd -

# Get the deploy key by using Travis's stored variables to decrypt deploy_key.enc
ENCRYPTION_LABEL=e3a2d77100be
ENCRYPTED_KEY_VAR="encrypted_${ENCRYPTION_LABEL}_key"
ENCRYPTED_IV_VAR="encrypted_${ENCRYPTION_LABEL}_iv"
ENCRYPTED_KEY=${!ENCRYPTED_KEY_VAR}
ENCRYPTED_IV=${!ENCRYPTED_IV_VAR}
openssl aes-256-cbc -K "$ENCRYPTED_KEY" -iv "$ENCRYPTED_IV" -in .github/deploy_key.enc -out .github/deploy_key -d
chmod 600 .github/deploy_key
eval "$(ssh-agent -s)"
ssh-add .github/deploy_key

cd out
git add .
git commit -m "Automatic deploy to GitHub Pages: ${SHA}"

# Now that we're all set up, we can push.
git push "$SSH_REPO" "$TARGET_BRANCH"
Binary file removed .github/deploy_key.enc
Binary file not shown.
29 changes: 29 additions & 0 deletions .github/driver.sh
Original file line number Diff line number Diff line change
@@ -0,0 +1,29 @@
#! /bin/bash

set -ex

# Check sysroot handling
sysroot=$(./target/debug/clippy-driver --print sysroot)
test "$sysroot" = "$(rustc --print sysroot)"

if [[ ${OS} == "Windows" ]]; then
desired_sysroot=C:/tmp
else
desired_sysroot=/tmp
fi
sysroot=$(./target/debug/clippy-driver --sysroot $desired_sysroot --print sysroot)
test "$sysroot" = $desired_sysroot

sysroot=$(SYSROOT=$desired_sysroot ./target/debug/clippy-driver --print sysroot)
test "$sysroot" = $desired_sysroot

# Make sure this isn't set - clippy-driver should cope without it
unset CARGO_MANIFEST_DIR

# Run a lint and make sure it produces the expected output. It's also expected to exit with code 1
# FIXME: How to match the clippy invocation in compile-test.rs?
./target/debug/clippy-driver -Dwarnings -Aunused -Zui-testing --emit metadata --crate-type bin tests/ui/cstring.rs 2> cstring.stderr && exit 1
sed -e "s,tests/ui,\$DIR," -e "/= help/d" cstring.stderr > normalized.stderr
diff normalized.stderr tests/ui/cstring.stderr

# TODO: CLIPPY_CONF_DIR / CARGO_MANIFEST_DIR
79 changes: 79 additions & 0 deletions .github/workflows/clippy.yml
Original file line number Diff line number Diff line change
@@ -0,0 +1,79 @@
name: Clippy Test

on:
push:
# Ignore bors branches, since they are covered by `clippy_bors.yml`
branches-ignore: [auto, try]
# Don't run Clippy tests, when only textfiles were modified
paths-ignore:
- 'COPYRIGHT'
- 'LICENSE-*'
- '**.md'
- '**.txt'
pull_request:
# Don't run Clippy tests, when only textfiles were modified
paths-ignore:
- 'COPYRIGHT'
- 'LICENSE-*'
- '**.md'
- '**.txt'

env:
RUST_BACKTRACE: 1
CARGO_TARGET_DIR: '${{ github.workspace }}/target'
GHA_CI: 1

jobs:
base:
runs-on: ubuntu-latest

steps:
- uses: rust-lang/simpleinfra/github-actions/cancel-outdated-builds@master
with:
github_token: "${{ secrets.github_token }}"
- name: rust-toolchain
uses: actions-rs/[email protected]
with:
toolchain: nightly
target: x86_64-unknown-linux-gnu
profile: minimal
- name: Cache cargo dir
uses: actions/cache@v1
with:
path: ~/.cargo
key: ${{ runner.os }}-x86_64-unknown-linux-gnu
- name: Checkout
uses: actions/[email protected]
- name: Master Toolchain Setup
run: bash setup-toolchain.sh

- name: Set LD_LIBRARY_PATH (Linux)
run: |
SYSROOT=$(rustc --print sysroot)
echo "::set-env name=LD_LIBRARY_PATH::${SYSROOT}/lib${LD_LIBRARY_PATH+:${LD_LIBRARY_PATH}}"
- name: Build
run: cargo build --features deny-warnings
- name: Test
run: cargo test --features deny-warnings
- name: Test clippy_lints
run: cargo test --features deny-warnings
working-directory: clippy_lints
- name: Test rustc_tools_util
run: cargo test --features deny-warnings
working-directory: rustc_tools_util
- name: Test clippy_dev
run: cargo test --features deny-warnings
working-directory: clippy_dev
- name: Test cargo-clippy
run: ../target/debug/cargo-clippy
working-directory: clippy_workspace_tests
- name: Test clippy-driver
run: bash .github/driver.sh
env:
OS: ${{ runner.os }}

- name: Run cargo-cache --autoclean
run: |
cargo install cargo-cache --debug
find ~/.cargo/bin ! -type d -exec strip {} \;
cargo cache --autoclean
Loading

0 comments on commit 62f1444

Please sign in to comment.