Asynchronous prefetching of data #1395
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| name: linting-and-tests | |
| on: [pull_request] | |
| defaults: | |
| run: | |
| shell: bash -l {0} | |
| jobs: | |
| flake8: | |
| timeout-minutes: 40 | |
| runs-on: ubuntu-latest | |
| steps: | |
| - name: Check out code | |
| uses: actions/checkout@v3 | |
| - name: Setup mamba | |
| uses: ./.github/actions/mamba | |
| - name: Flake8 | |
| run: | | |
| micromamba run -n modyn flake8 --version | |
| micromamba run -n modyn flake8 modyn --statistics | |
| mypy-typechecking: | |
| timeout-minutes: 20 | |
| runs-on: ubuntu-latest | |
| steps: | |
| - name: Check out code | |
| uses: actions/checkout@v3 | |
| - name: Setup mamba | |
| uses: ./.github/actions/mamba | |
| - name: Mypy | |
| run: | | |
| micromamba run -n modyn mypy --version | |
| micromamba run -n modyn mypy modyn | |
| pylint: | |
| timeout-minutes: 20 | |
| runs-on: ubuntu-latest | |
| steps: | |
| - name: Check out code | |
| uses: actions/checkout@v3 | |
| - name: Setup mamba | |
| uses: ./.github/actions/mamba | |
| - name: Pylint | |
| run: | | |
| micromamba run -n modyn pylint --version | |
| micromamba run -n modyn pylint modyn | |
| isort: | |
| timeout-minutes: 20 | |
| runs-on: ubuntu-latest | |
| steps: | |
| - name: Check out code | |
| uses: actions/checkout@v3 | |
| - name: Setup mamba | |
| uses: ./.github/actions/mamba | |
| - name: Isort | |
| run: | | |
| micromamba run -n modyn isort --version | |
| micromamba run -n modyn isort modyn --check --diff | |
| micromamba run -n modyn isort integrationtests --check --diff | |
| micromamba run -n modyn isort benchmark --check --diff | |
| black: | |
| timeout-minutes: 20 | |
| runs-on: ubuntu-latest | |
| steps: | |
| - name: Check out code | |
| uses: actions/checkout@v3 | |
| - name: Setup mamba | |
| uses: ./.github/actions/mamba | |
| - name: Black | |
| run: | | |
| micromamba run -n modyn black --version | |
| micromamba run -n modyn black --check modyn --verbose --config black.toml | |
| unittests: | |
| timeout-minutes: 20 | |
| runs-on: ubuntu-latest | |
| steps: | |
| - name: Check out code | |
| uses: actions/checkout@v3 | |
| - name: Setup mamba | |
| uses: ./.github/actions/mamba | |
| - name: Pytest | |
| run: | | |
| micromamba run -n modyn pytest modyn --cov-reset --cache-clear --cov-fail-under=90 | |
| micromamba run -n modyn pytest > pytest-coverage.txt | |
| - name: Comment coverage | |
| uses: coroo/[email protected] | |
| ### Integration Tests ### | |
| # We have them in the same workflow because it's impossible to have a simple "if workflow A runs through completely, then workflow B should run" pipeline on Github currently | |
| # Checks whether the base container works correctly. | |
| dockerized-unittests: | |
| timeout-minutes: 60 | |
| runs-on: ubuntu-latest | |
| needs: | |
| - flake8 | |
| - mypy-typechecking | |
| - pylint | |
| - unittests | |
| - isort | |
| - black | |
| steps: | |
| - name: Check out code | |
| uses: actions/checkout@v3 | |
| - name: Setup base container | |
| uses: ./.github/actions/base | |
| - name: Setup dev-requirements and run pytest within container | |
| run: docker run modynbase mamba run -n modyn bash -c "pip install -r dev-requirements.txt && echo Running pytest && pytest" | |
| # Tests whether docker-compose up starts all components successfully and integration tests run through | |
| # Only one job to reduce Github CI usage | |
| integrationtests: | |
| timeout-minutes: 60 | |
| runs-on: ubuntu-latest | |
| needs: | |
| - flake8 | |
| - mypy-typechecking | |
| - pylint | |
| - unittests | |
| - isort | |
| - black | |
| - dockerized-unittests | |
| steps: | |
| - name: Check out code | |
| uses: actions/checkout@v3 | |
| - name: Start docker compose and exit when tests run through | |
| run: bash run_integrationtests.sh |