Skip to content

Commit

Permalink
Testing build only on successful tests
Browse files Browse the repository at this point in the history
Github actions didn't have out of the box solution for running a workflow based on results of multiple workflows where ALL workflows must have completed successfully.
We need this since both the test-with-docker and test-with-manual-install must pass.
So this needs an "AND" logic.

"workflow_run" is there but this triggers the dependent workflow when either of the workflow dependencies defined as prerequisites are completed.
So this has an "OR" logic.

----

Found an alternative suggestion here:
https://stackoverflow.com/a/75597437

This suggests converting the pre-requisite workflows into reusable workflows.
These workflows can then be called in jobs in the workflow that needs these workflows to be run before.
Finally, these jobs can be added as dependencies for the requisite job.

In our scenario, two new jobs are added to the image_build_push.yml for each of the two tests environments.
These will run parallelly.

Then in the build image job, these jobs are added in the "needs" field, indicating that these jobs must pass successfully before running the build job.
  • Loading branch information
MukuFlash03 committed Sep 29, 2024
1 parent 8adfed0 commit c2f2ed8
Show file tree
Hide file tree
Showing 3 changed files with 142 additions and 99 deletions.
67 changes: 38 additions & 29 deletions .github/workflows/image_build_push.yml
Original file line number Diff line number Diff line change
Expand Up @@ -3,6 +3,8 @@ name: docker image
on:
push:
branches: [ master, gis-based-mode-detection, cleanup-cicd ]
pull_request:
branches: [ master, gis-based-mode-detection ]

# Dockerhub credentials are set as environment variables
env:
Expand Down Expand Up @@ -49,8 +51,15 @@ jobs:
echo "Event name: ${{ github.event_name }}"
echo "Actor: ${{ github.actor }}"
test-with-docker:
uses: MukuFlash03/e-mission-server/.github/workflows/test-with-docker.yml@cleanup-cicd

test-with-manual-install:
uses: MukuFlash03/e-mission-server/.github/workflows/test-with-manual-install.yml@cleanup-cicd

build:
runs-on: ubuntu-latest
needs: [test-with-docker, test-with-manual-install]

outputs:
date: ${{ steps.date.outputs.date }}
Expand Down Expand Up @@ -103,36 +112,36 @@ jobs:
git push origin
fi
dispatch:
needs: build
runs-on: ubuntu-latest

strategy:
matrix:
include:
- repo: MukuFlash03/nrel-openpath-join-page
branch: cleanup-cicd
- repo: MukuFlash03/op-admin-dashboard
branch: cleanup-cicd
- repo: MukuFlash03/em-public-dashboard
branch: cleanup-cicd
# - repo: e-mission/op-admin-dashboard
# branch: master
# - repo: e-mission/em-public-dashboard
# branch: main

steps:
- uses: actions/checkout@v4
# dispatch:
# needs: build
# runs-on: ubuntu-latest

- name: Trigger workflow in admin-dash, public-dash
run: |
curl -L \
-X POST \
-H "Accept: application/vnd.github+json" \
-H "Authorization: Bearer ${{ secrets.GH_FG_PAT_TAGS }}" \
-H "X-GitHub-Api-Version: 2022-11-28" \
https://api.github.com/repos/${{ matrix.repo }}/actions/workflows/image_build_push.yml/dispatches \
-d '{"ref":"${{ matrix.branch }}"}'
# strategy:
# matrix:
# include:
# - repo: MukuFlash03/nrel-openpath-join-page
# branch: cleanup-cicd
# - repo: MukuFlash03/op-admin-dashboard
# branch: cleanup-cicd
# - repo: MukuFlash03/em-public-dashboard
# branch: cleanup-cicd
# # - repo: e-mission/op-admin-dashboard
# # branch: master
# # - repo: e-mission/em-public-dashboard
# # branch: main

# steps:
# - uses: actions/checkout@v4

# - name: Trigger workflow in admin-dash, public-dash
# run: |
# curl -L \
# -X POST \
# -H "Accept: application/vnd.github+json" \
# -H "Authorization: Bearer ${{ secrets.GH_FG_PAT_TAGS }}" \
# -H "X-GitHub-Api-Version: 2022-11-28" \
# https://api.github.com/repos/${{ matrix.repo }}/actions/workflows/image_build_push.yml/dispatches \
# -d '{"ref":"${{ matrix.branch }}"}'


# cascade-image-build:
Expand Down
51 changes: 34 additions & 17 deletions .github/workflows/test-with-docker.yml
Original file line number Diff line number Diff line change
Expand Up @@ -5,29 +5,46 @@ name: test-with-docker
# Controls when the action will run. Triggers the workflow on push or pull request
# events but only for the master branch
on:
push:
branches: [ master ]
pull_request:
branches: [ master ]
# push:
# branches: [ master ]
# pull_request:
# branches: [ master ]
schedule:
# * is a special character in YAML so you have to quote this string
- cron: '5 4 * * 0'

# A workflow run is made up of one or more jobs that can run sequentially or in parallel
jobs:
# This workflow contains a single job called "build"
workflow_call:

jobs:
build:
# The type of runner that the job will run on
runs-on: ubuntu-latest

# Steps represent a sequence of tasks that will be executed as part of the job
steps:
# Checks-out your repository under $GITHUB_WORKSPACE, so your job can access it
- name: Checkout
uses: actions/checkout@v2
- name: Run test-with-docker for 30 seconds
run: |
echo "Starting 1-minute workflow"
sleep 30
echo "30-second workflow completed"
- name: Success-Failure Test Case
run: |
echo "Test successful" && exit 0
# echo "Test failed" && exit 1
# # A workflow run is made up of one or more jobs that can run sequentially or in parallel
# jobs:
# # This workflow contains a single job called "build"
# build:
# # The type of runner that the job will run on
# runs-on: ubuntu-latest

# # Steps represent a sequence of tasks that will be executed as part of the job
# steps:
# # Checks-out your repository under $GITHUB_WORKSPACE, so your job can access it
# - name: Checkout
# uses: actions/checkout@v2

- name: Make sure that the workflow works
run: echo Smoke test
# - name: Make sure that the workflow works
# run: echo Smoke test

- name: Run the tests using docker-compose
run: docker compose -f setup/docker-compose.tests.yml up --exit-code-from web-server
# - name: Run the tests using docker-compose
# run: docker compose -f setup/docker-compose.tests.yml up --exit-code-from web-server
123 changes: 70 additions & 53 deletions .github/workflows/test-with-manual-install.yml
Original file line number Diff line number Diff line change
Expand Up @@ -5,69 +5,86 @@ name: ubuntu-only-test-with-manual-install
# Controls when the action will run. Triggers the workflow on push or pull request
# events but only for the master branch
on:
push:
branches:
- master
- gis-based-mode-detection
pull_request:
branches:
- master
- gis-based-mode-detection
# push:
# branches:
# - master
# - gis-based-mode-detection
# pull_request:
# branches:
# - master
# - gis-based-mode-detection
schedule:
# * is a special character in YAML so you have to quote this string
- cron: '5 4 * * 0'

# A workflow run is made up of one or more jobs that can run sequentially or in parallel
jobs:
# This workflow contains a single job called "build"
build:
# The type of runner that the job will run on
runs-on: ${{ matrix.os }}
strategy:
matrix:
os: [ubuntu-latest]
workflow_call:

# Steps represent a sequence of tasks that will be executed as part of the job
jobs:
build:
runs-on: ubuntu-latest
steps:
# Checks-out your repository under $GITHUB_WORKSPACE, so your job can access it
- uses: actions/checkout@v2
- name: Run test-with-manual-install for 1 minute
run: |
echo "Starting 1-minute workflow"
sleep 60
echo "1-minute workflow completed"
- name: Success-Failure Test Case
run: |
echo "Test successful" && exit 0
# echo "Test failed" && exit 1
# # A workflow run is made up of one or more jobs that can run sequentially or in parallel
# jobs:
# # This workflow contains a single job called "build"
# build:
# # The type of runner that the job will run on
# runs-on: ${{ matrix.os }}
# strategy:
# matrix:
# os: [ubuntu-latest]

# # Steps represent a sequence of tasks that will be executed as part of the job
# steps:
# # Checks-out your repository under $GITHUB_WORKSPACE, so your job can access it
# - uses: actions/checkout@v2

- name: Install and start MongoDB
uses: supercharge/[email protected]
with:
mongodb-version: 4.4.0
# - name: Install and start MongoDB
# uses: supercharge/[email protected]
# with:
# mongodb-version: 4.4.0

- name: Check existing version of miniconda
shell: bash -l {0}
run: conda info -a
# - name: Check existing version of miniconda
# shell: bash -l {0}
# run: conda info -a

- name: Install miniconda
shell: bash -l {0}
run: |
source setup/setup_conda.sh Linux-x86_64
# - name: Install miniconda
# shell: bash -l {0}
# run: |
# source setup/setup_conda.sh Linux-x86_64

- name: Check whether the CI environment variable is set
shell: bash -l {0}
run: |
source setup/activate_conda.sh
echo $CI
# - name: Check whether the CI environment variable is set
# shell: bash -l {0}
# run: |
# source setup/activate_conda.sh
# echo $CI

- name: Setup the emission environment for testing
shell: bash -l {0}
run: |
conda --version
which conda
source setup/activate_conda.sh
conda --version
source setup/setup_tests.sh
# - name: Setup the emission environment for testing
# shell: bash -l {0}
# run: |
# conda --version
# which conda
# source setup/activate_conda.sh
# conda --version
# source setup/setup_tests.sh

- name: Switch to emission and run the tests
shell: bash -l {0}
run: |
source setup/activate_tests.sh
conda --version
./runAllTests.sh
# - name: Switch to emission and run the tests
# shell: bash -l {0}
# run: |
# source setup/activate_tests.sh
# conda --version
# ./runAllTests.sh

- name: Teardown the test environment
shell: bash -l {0}
run: source setup/teardown_tests.sh
# - name: Teardown the test environment
# shell: bash -l {0}
# run: source setup/teardown_tests.sh

0 comments on commit c2f2ed8

Please sign in to comment.