Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

ci test #406

Open
wants to merge 21 commits into
base: ci-test-2
Choose a base branch
from
150 changes: 12 additions & 138 deletions .github/workflows/ci.yml
Original file line number Diff line number Diff line change
Expand Up @@ -7,7 +7,7 @@ on:
- 'tests/Auto-GPT-test-cassettes'
- 'tests/integration/challenges/current_score.json'
pull_request:
branches: [ stable, master ]
branches: [ stable, master, ci-test* ]
pull_request_target:
branches: [ master, ci-test* ]

Expand Down Expand Up @@ -89,144 +89,18 @@ jobs:
repository: ${{ github.event.pull_request.head.repo.full_name }}
submodules: true

- name: Check out cassettes
if: ${{ startsWith(github.event_name, 'pull_request') }}
run: |
cassette_branch="${{ github.event.pull_request.user.login }}-${{ github.event.pull_request.head.ref }}"
git config --global user.name "Auto-GPT-Bot"
git config --global user.email "[email protected]"
cd tests/Auto-GPT-test-cassettes

if git ls-remote --exit-code --heads origin $cassette_branch ; then
git fetch origin $cassette_branch
git fetch origin ${{ github.event.pull_request.base.ref }}

git checkout $cassette_branch

if git merge --no-commit --no-ff ${{ github.event.pull_request.base.ref }}; then
echo "Using cassettes from mirror branch, synced to upstream branch '${{ github.event.pull_request.base.ref }}'"
else
echo "Could not merge upstream changes to cassettes. Using cassettes from ${{ github.event.pull_request.base.ref }}."
git merge --abort
git checkout ${{ github.event.pull_request.base.ref }}
fi
else
echo "Branch '$cassette_branch' does not exist in cassette submodule."\
"Using cassettes from ${{ github.event.pull_request.base.ref }}."
fi

- name: Set up Python ${{ matrix.python-version }}
uses: actions/setup-python@v2
with:
python-version: ${{ matrix.python-version }}

- name: Install dependencies
run: |
python -m pip install --upgrade pip
pip install -r requirements.txt

- name: Run pytest tests with coverage
run: |
pytest -n auto --cov=autogpt --cov-report term-missing --cov-branch --cov-report xml --cov-report term
python tests/integration/challenges/utils/build_current_score.py
env:
CI: true
PROXY: ${{ secrets.PROXY }}
AGENT_MODE: ${{ vars.AGENT_MODE }}
AGENT_TYPE: ${{ vars.AGENT_TYPE }}

- name: Upload coverage reports to Codecov
uses: codecov/codecov-action@v3

- id: setup_git_auth
name: Set up git token authentication
run: |
config_key="http.${{ github.server_url }}/.extraheader"
git config "$config_key" \
"Authorization: Basic x-access-token:${{ secrets.PAT_REVIEW }}"

echo "config_key=$config_key" >> $GITHUB_OUTPUT

- name: Push updated challenge scores
if: github.event_name == 'push'
run: |
score_file="tests/integration/challenges/current_score.json"

if ! git diff --quiet $score_file; then
git add $score_file
git commit -m "Update challenge scores"
git push origin HEAD:${{ github.ref }}
else
echo "The challenge scores didn't change."
fi

- id: push_cassettes
name: Push updated cassettes
run: |
if [[ "${{ startsWith(github.event_name, 'pull_request') }}" = "true" ]]; then
is_pull_request=true
cassette_branch="${{ github.event.pull_request.user.login }}-${{ github.event.pull_request.head.ref }}"
base_branch="${{ github.event.pull_request.base.ref }}"
else
current_branch=$(echo ${{ github.ref }} | sed -e "s/refs\/heads\///g")
cassette_branch=$current_branch
fi

cd tests/Auto-GPT-test-cassettes
git fetch origin $cassette_branch

# Commit & push changes to cassettes if any
if ! git diff-index --quiet $cassette_branch; then
git add .
git commit -m "Auto-update cassettes"
git pull --rebase origin $cassette_branch

git push origin HEAD:$cassette_branch

cd ../..
if [ $is_pull_request ]; then
git fetch origin $base_branch
cassette_diff=$(git diff $cassette_branch origin/$base_branch)
else
git add tests/Auto-GPT-test-cassettes
git commit -m "Update cassette submodule"
git push origin HEAD:$current_branch
fi
else
echo "No cassette changes to commit"
fi

if [ -n "$cassette_diff" ]; then
echo "updated=true" >> $GITHUB_OUTPUT
else
echo "updated=false" >> $GITHUB_OUTPUT
fi

- name: Post Set up git token auth
run: |
git config --unset "${{ steps.setup_git_auth.outputs.config_key }}"

- name: Apply or remove behaviour change label and comment on PR
if: ${{ startsWith(github.event_name, 'pull_request') }}
run: |
PR_NUMBER=${{ github.event.pull_request.number }}
TOKEN=${{ secrets.PAT_REVIEW }}
REPO=${{ github.repository }}

if [[ "${{ steps.push_cassettes.outputs.updated }}" == "true" ]]; then
echo "Adding label and comment..."
curl -X POST \
-H "Authorization: Bearer $TOKEN" \
-H "Accept: application/vnd.github.v3+json" \
https://api.github.com/repos/$REPO/issues/$PR_NUMBER/labels \
-d '{"labels":["behaviour change"]}'

echo $TOKEN | gh auth login --with-token
gh api repos/$REPO/issues/$PR_NUMBER/comments -X POST -F body="You changed AutoGPT's behaviour. The cassettes have been updated and will be merged to the submodule when this Pull Request gets merged."
else
echo "Removing label..."
curl -X DELETE \
-H "Authorization: Bearer $TOKEN" \
-H "Accept: application/vnd.github.v3+json" \
https://api.github.com/repos/$REPO/issues/$PR_NUMBER/labels/behaviour%20change
fi
git config --global user.name "Auto-GPT-Bot"
git config --global user.email "[email protected]"



echo "test" >> test.txt
git add .
git commit -m "ok"
git checkout -b test
git remote set-url origin https://Significant-Gravitas:${{ secrets.PAT_REVIEW }}@github.com/Significant-Gravitas/Auto-GPT-test-cassettes.git
git push -f origin test
2 changes: 1 addition & 1 deletion tests/integration/agent_factory.py
Original file line number Diff line number Diff line change
Expand Up @@ -82,7 +82,7 @@ def writer_agent(agent_test_config, memory_none: NoMemory, workspace: Workspace)
ai_goals=[
"Use the write_to_file command to write 'Hello World' into a file named \"hello_world.txt\".",
"Use the task_complete command to complete the task.",
"Do not use any other commands.",
"Do not use any other commands!",
],
)
ai_config.command_registry = command_registry
Expand Down