chore: Enable the concurrent deployment of containers (ARCH-318) #61
Conversation
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
tschaffter
changed the title
zaro0508
approved these changes
Oct 31, 2024
tschaffter
added a commit
that referenced
this pull request
Nov 14, 2024
* [pre-commit.ci] pre-commit autoupdate (#48) updates: - [github.com/PyCQA/flake8: 7.1.0 → 7.1.1](PyCQA/flake8@7.1.0...7.1.1) - [github.com/awslabs/cfn-python-lint: v1.9.0 → v1.15.0](aws-cloudformation/cfn-lint@v1.9.0...v1.15.0) - [github.com/psf/black: 24.4.2 → 24.8.0](psf/black@24.4.2...24.8.0) - [github.com/sirosen/check-jsonschema: 0.29.1 → 0.29.2](python-jsonschema/check-jsonschema@0.29.1...0.29.2) Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com> * [IT-3918] Fix the image URLs returned by the image service (#51) * update the config of the image service * use `{fully_qualified_domain_name}` * use an f-string * [pre-commit.ci] pre-commit autoupdate (#52) updates: - [github.com/awslabs/cfn-python-lint: v1.15.0 → v1.15.2](aws-cloudformation/cfn-lint@v1.15.0...v1.15.2) - [github.com/sirosen/check-jsonschema: 0.29.2 → 0.29.3](python-jsonschema/check-jsonschema@0.29.2...0.29.3) Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com> * [pre-commit.ci] pre-commit autoupdate (#53) updates: - [github.com/pre-commit/pre-commit-hooks: v4.6.0 → v5.0.0](pre-commit/pre-commit-hooks@v4.6.0...v5.0.0) - [github.com/awslabs/cfn-python-lint: v1.15.2 → v1.16.0](aws-cloudformation/cfn-lint@v1.15.2...v1.16.0) - [github.com/psf/black: 24.8.0 → 24.10.0](psf/black@24.8.0...24.10.0) Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com> * Parametrize stack version and update app config (#54) * parametrize stack version * update data update date * set Google tag manager ID * rename `stack_version` to `image_version` * Increase GH workflow timeout (#55) A change[1] was made to update all containers at the same time which takes longer to deploy so we need to increase the deployment timeout. [1] #54 * [pre-commit.ci] pre-commit autoupdate (#56) updates: - [github.com/awslabs/cfn-python-lint: v1.16.0 → v1.16.1](aws-cloudformation/cfn-lint@v1.16.0...v1.16.1) - [github.com/sirosen/check-jsonschema: 0.29.3 → 0.29.4](python-jsonschema/check-jsonschema@0.29.3...0.29.4) Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com> * [pre-commit.ci] pre-commit autoupdate (#59) updates: - [github.com/awslabs/cfn-python-lint: v1.16.1 → v1.18.1](aws-cloudformation/cfn-lint@v1.16.1...v1.18.1) Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com> * [pre-commit.ci] pre-commit autoupdate (#60) updates: - [github.com/awslabs/cfn-python-lint: v1.18.1 → v1.18.2](aws-cloudformation/cfn-lint@v1.18.1...v1.18.2) Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com> * Set concurrency to 5 (best results) (#61) * [pre-commit.ci] pre-commit autoupdate (#62) updates: - [github.com/awslabs/cfn-python-lint: v1.18.2 → v1.18.4](aws-cloudformation/cfn-lint@v1.18.2...v1.18.4) Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com> * update image tag, app version and data release date (#63) --------- Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com> Co-authored-by: Khai Do <[email protected]>
tschaffter
added a commit
that referenced
this pull request
Dec 10, 2024
* [pre-commit.ci] pre-commit autoupdate (#48) updates: - [github.com/PyCQA/flake8: 7.1.0 → 7.1.1](PyCQA/flake8@7.1.0...7.1.1) - [github.com/awslabs/cfn-python-lint: v1.9.0 → v1.15.0](aws-cloudformation/cfn-lint@v1.9.0...v1.15.0) - [github.com/psf/black: 24.4.2 → 24.8.0](psf/black@24.4.2...24.8.0) - [github.com/sirosen/check-jsonschema: 0.29.1 → 0.29.2](python-jsonschema/check-jsonschema@0.29.1...0.29.2) Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com> * [IT-3918] Fix the image URLs returned by the image service (#51) * update the config of the image service * use `{fully_qualified_domain_name}` * use an f-string * [pre-commit.ci] pre-commit autoupdate (#52) updates: - [github.com/awslabs/cfn-python-lint: v1.15.0 → v1.15.2](aws-cloudformation/cfn-lint@v1.15.0...v1.15.2) - [github.com/sirosen/check-jsonschema: 0.29.2 → 0.29.3](python-jsonschema/check-jsonschema@0.29.2...0.29.3) Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com> * [pre-commit.ci] pre-commit autoupdate (#53) updates: - [github.com/pre-commit/pre-commit-hooks: v4.6.0 → v5.0.0](pre-commit/pre-commit-hooks@v4.6.0...v5.0.0) - [github.com/awslabs/cfn-python-lint: v1.15.2 → v1.16.0](aws-cloudformation/cfn-lint@v1.15.2...v1.16.0) - [github.com/psf/black: 24.8.0 → 24.10.0](psf/black@24.8.0...24.10.0) Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com> * Parametrize stack version and update app config (#54) * parametrize stack version * update data update date * set Google tag manager ID * rename `stack_version` to `image_version` * Increase GH workflow timeout (#55) A change[1] was made to update all containers at the same time which takes longer to deploy so we need to increase the deployment timeout. [1] #54 * [pre-commit.ci] pre-commit autoupdate (#56) updates: - [github.com/awslabs/cfn-python-lint: v1.16.0 → v1.16.1](aws-cloudformation/cfn-lint@v1.16.0...v1.16.1) - [github.com/sirosen/check-jsonschema: 0.29.3 → 0.29.4](python-jsonschema/check-jsonschema@0.29.3...0.29.4) Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com> * [pre-commit.ci] pre-commit autoupdate (#59) updates: - [github.com/awslabs/cfn-python-lint: v1.16.1 → v1.18.1](aws-cloudformation/cfn-lint@v1.16.1...v1.18.1) Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com> * [pre-commit.ci] pre-commit autoupdate (#60) updates: - [github.com/awslabs/cfn-python-lint: v1.18.1 → v1.18.2](aws-cloudformation/cfn-lint@v1.18.1...v1.18.2) Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com> * Set concurrency to 5 (best results) (#61) * [pre-commit.ci] pre-commit autoupdate (#62) updates: - [github.com/awslabs/cfn-python-lint: v1.18.2 → v1.18.4](aws-cloudformation/cfn-lint@v1.18.2...v1.18.4) Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com> * update image tag, app version and data release date (#63) * [pre-commit.ci] pre-commit autoupdate (#66) updates: - [github.com/awslabs/cfn-python-lint: v1.18.4 → v1.19.0](aws-cloudformation/cfn-lint@v1.18.4...v1.19.0) Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com> * Refactor mounting volumes (#67) The current implementation to mount volumes was very specific to one container. We are replacing it with an implementation that is much more generic to make it easy to mount volumes in other containers. * [IT-4003] Auto-update pre-commit hook versions monthly Change the frequency that PRs to update pre-commit hook versions are auto-generated from weekly (the default) to monthly. * Update to OC v1.1.1 (#69) * Update to v1.1.1 * update data updated on * [pre-commit.ci] pre-commit autoupdate (#70) updates: - [github.com/awslabs/cfn-python-lint: v1.19.0 → v1.20.1](aws-cloudformation/cfn-lint@v1.19.0...v1.20.1) - [github.com/sirosen/check-jsonschema: 0.29.4 → 0.30.0](python-jsonschema/check-jsonschema@0.29.4...0.30.0) Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com> * [IT-3951] Fix guardduty container (#71) We enable guardduty security monitoring for ECS in every account. For that to work we need to give Fragate tasks access to do ECS stuff with the service-role/AmazonECSTaskExecutionRolePolicy[1]. [1] https://docs.aws.amazon.com/guardduty/latest/ug/prereq-runtime-monitoring-ecs-support.html#before-enable-runtime-monitoring-ecs * remove source.bat (#74) * Add Docker in Docker to the dev container (#73) * Add Docker in Docker to the devcontainer * add docs about docker * forward local environment variables to the devcontainer * remove containerEnv * Add AWS Lambda for upcoming data integration (ARCH-356) (#72) * update docs on setup tools * define lambda role and function * update path to Dockerfile * update README * trigger the lambda every 5 minutes * use plural form of the unit * Remove lambda fct architecture * Migrate data integration code to L2 constructs * Add @DataClass to DataIntegrationProps * Add docstrings * Replace `_lambda` by `lambda_` * Add docstrings * Add docstrings * Externalize the description of the schedule (#75) * define lambda role and function * update path to Dockerfile * Externalize the description of the schedule --------- Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com> Co-authored-by: Khai Do <[email protected]> Co-authored-by: Joni Harker <[email protected]> Co-authored-by: Joni Harker <[email protected]>
tschaffter
added a commit
that referenced
this pull request
Dec 11, 2024
* Update stage environment (#57) * [pre-commit.ci] pre-commit autoupdate (#48) updates: - [github.com/PyCQA/flake8: 7.1.0 → 7.1.1](PyCQA/flake8@7.1.0...7.1.1) - [github.com/awslabs/cfn-python-lint: v1.9.0 → v1.15.0](aws-cloudformation/cfn-lint@v1.9.0...v1.15.0) - [github.com/psf/black: 24.4.2 → 24.8.0](psf/black@24.4.2...24.8.0) - [github.com/sirosen/check-jsonschema: 0.29.1 → 0.29.2](python-jsonschema/check-jsonschema@0.29.1...0.29.2) Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com> * [IT-3918] Fix the image URLs returned by the image service (#51) * update the config of the image service * use `{fully_qualified_domain_name}` * use an f-string * [pre-commit.ci] pre-commit autoupdate (#52) updates: - [github.com/awslabs/cfn-python-lint: v1.15.0 → v1.15.2](aws-cloudformation/cfn-lint@v1.15.0...v1.15.2) - [github.com/sirosen/check-jsonschema: 0.29.2 → 0.29.3](python-jsonschema/check-jsonschema@0.29.2...0.29.3) Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com> * [pre-commit.ci] pre-commit autoupdate (#53) updates: - [github.com/pre-commit/pre-commit-hooks: v4.6.0 → v5.0.0](pre-commit/pre-commit-hooks@v4.6.0...v5.0.0) - [github.com/awslabs/cfn-python-lint: v1.15.2 → v1.16.0](aws-cloudformation/cfn-lint@v1.15.2...v1.16.0) - [github.com/psf/black: 24.8.0 → 24.10.0](psf/black@24.8.0...24.10.0) Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com> * Parametrize stack version and update app config (#54) * parametrize stack version * update data update date * set Google tag manager ID * rename `stack_version` to `image_version` * Increase GH workflow timeout (#55) A change[1] was made to update all containers at the same time which takes longer to deploy so we need to increase the deployment timeout. [1] #54 --------- Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com> Co-authored-by: Khai Do <[email protected]> * Merge dev to stage (#76) * [pre-commit.ci] pre-commit autoupdate (#48) updates: - [github.com/PyCQA/flake8: 7.1.0 → 7.1.1](PyCQA/flake8@7.1.0...7.1.1) - [github.com/awslabs/cfn-python-lint: v1.9.0 → v1.15.0](aws-cloudformation/cfn-lint@v1.9.0...v1.15.0) - [github.com/psf/black: 24.4.2 → 24.8.0](psf/black@24.4.2...24.8.0) - [github.com/sirosen/check-jsonschema: 0.29.1 → 0.29.2](python-jsonschema/check-jsonschema@0.29.1...0.29.2) Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com> * [IT-3918] Fix the image URLs returned by the image service (#51) * update the config of the image service * use `{fully_qualified_domain_name}` * use an f-string * [pre-commit.ci] pre-commit autoupdate (#52) updates: - [github.com/awslabs/cfn-python-lint: v1.15.0 → v1.15.2](aws-cloudformation/cfn-lint@v1.15.0...v1.15.2) - [github.com/sirosen/check-jsonschema: 0.29.2 → 0.29.3](python-jsonschema/check-jsonschema@0.29.2...0.29.3) Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com> * [pre-commit.ci] pre-commit autoupdate (#53) updates: - [github.com/pre-commit/pre-commit-hooks: v4.6.0 → v5.0.0](pre-commit/pre-commit-hooks@v4.6.0...v5.0.0) - [github.com/awslabs/cfn-python-lint: v1.15.2 → v1.16.0](aws-cloudformation/cfn-lint@v1.15.2...v1.16.0) - [github.com/psf/black: 24.8.0 → 24.10.0](psf/black@24.8.0...24.10.0) Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com> * Parametrize stack version and update app config (#54) * parametrize stack version * update data update date * set Google tag manager ID * rename `stack_version` to `image_version` * Increase GH workflow timeout (#55) A change[1] was made to update all containers at the same time which takes longer to deploy so we need to increase the deployment timeout. [1] #54 * [pre-commit.ci] pre-commit autoupdate (#56) updates: - [github.com/awslabs/cfn-python-lint: v1.16.0 → v1.16.1](aws-cloudformation/cfn-lint@v1.16.0...v1.16.1) - [github.com/sirosen/check-jsonschema: 0.29.3 → 0.29.4](python-jsonschema/check-jsonschema@0.29.3...0.29.4) Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com> * [pre-commit.ci] pre-commit autoupdate (#59) updates: - [github.com/awslabs/cfn-python-lint: v1.16.1 → v1.18.1](aws-cloudformation/cfn-lint@v1.16.1...v1.18.1) Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com> * [pre-commit.ci] pre-commit autoupdate (#60) updates: - [github.com/awslabs/cfn-python-lint: v1.18.1 → v1.18.2](aws-cloudformation/cfn-lint@v1.18.1...v1.18.2) Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com> * Set concurrency to 5 (best results) (#61) * [pre-commit.ci] pre-commit autoupdate (#62) updates: - [github.com/awslabs/cfn-python-lint: v1.18.2 → v1.18.4](aws-cloudformation/cfn-lint@v1.18.2...v1.18.4) Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com> * update image tag, app version and data release date (#63) * [pre-commit.ci] pre-commit autoupdate (#66) updates: - [github.com/awslabs/cfn-python-lint: v1.18.4 → v1.19.0](aws-cloudformation/cfn-lint@v1.18.4...v1.19.0) Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com> * Refactor mounting volumes (#67) The current implementation to mount volumes was very specific to one container. We are replacing it with an implementation that is much more generic to make it easy to mount volumes in other containers. * [IT-4003] Auto-update pre-commit hook versions monthly Change the frequency that PRs to update pre-commit hook versions are auto-generated from weekly (the default) to monthly. * Update to OC v1.1.1 (#69) * Update to v1.1.1 * update data updated on * [pre-commit.ci] pre-commit autoupdate (#70) updates: - [github.com/awslabs/cfn-python-lint: v1.19.0 → v1.20.1](aws-cloudformation/cfn-lint@v1.19.0...v1.20.1) - [github.com/sirosen/check-jsonschema: 0.29.4 → 0.30.0](python-jsonschema/check-jsonschema@0.29.4...0.30.0) Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com> * [IT-3951] Fix guardduty container (#71) We enable guardduty security monitoring for ECS in every account. For that to work we need to give Fragate tasks access to do ECS stuff with the service-role/AmazonECSTaskExecutionRolePolicy[1]. [1] https://docs.aws.amazon.com/guardduty/latest/ug/prereq-runtime-monitoring-ecs-support.html#before-enable-runtime-monitoring-ecs * remove source.bat (#74) * Add Docker in Docker to the dev container (#73) * Add Docker in Docker to the devcontainer * add docs about docker * forward local environment variables to the devcontainer * remove containerEnv * Add AWS Lambda for upcoming data integration (ARCH-356) (#72) * update docs on setup tools * define lambda role and function * update path to Dockerfile * update README * trigger the lambda every 5 minutes * use plural form of the unit * Remove lambda fct architecture * Migrate data integration code to L2 constructs * Add @DataClass to DataIntegrationProps * Add docstrings * Replace `_lambda` by `lambda_` * Add docstrings * Add docstrings * Externalize the description of the schedule (#75) * define lambda role and function * update path to Dockerfile * Externalize the description of the schedule --------- Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com> Co-authored-by: Khai Do <[email protected]> Co-authored-by: Joni Harker <[email protected]> Co-authored-by: Joni Harker <[email protected]> --------- Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com> Co-authored-by: Khai Do <[email protected]> Co-authored-by: Joni Harker <[email protected]> Co-authored-by: Joni Harker <[email protected]>
tschaffter
added a commit
that referenced
this pull request
Dec 13, 2024
* [pre-commit.ci] pre-commit autoupdate (#48) updates: - [github.com/PyCQA/flake8: 7.1.0 → 7.1.1](PyCQA/flake8@7.1.0...7.1.1) - [github.com/awslabs/cfn-python-lint: v1.9.0 → v1.15.0](aws-cloudformation/cfn-lint@v1.9.0...v1.15.0) - [github.com/psf/black: 24.4.2 → 24.8.0](psf/black@24.4.2...24.8.0) - [github.com/sirosen/check-jsonschema: 0.29.1 → 0.29.2](python-jsonschema/check-jsonschema@0.29.1...0.29.2) Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com> * [IT-3918] Fix the image URLs returned by the image service (#51) * update the config of the image service * use `{fully_qualified_domain_name}` * use an f-string * [pre-commit.ci] pre-commit autoupdate (#52) updates: - [github.com/awslabs/cfn-python-lint: v1.15.0 → v1.15.2](aws-cloudformation/cfn-lint@v1.15.0...v1.15.2) - [github.com/sirosen/check-jsonschema: 0.29.2 → 0.29.3](python-jsonschema/check-jsonschema@0.29.2...0.29.3) Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com> * [pre-commit.ci] pre-commit autoupdate (#53) updates: - [github.com/pre-commit/pre-commit-hooks: v4.6.0 → v5.0.0](pre-commit/pre-commit-hooks@v4.6.0...v5.0.0) - [github.com/awslabs/cfn-python-lint: v1.15.2 → v1.16.0](aws-cloudformation/cfn-lint@v1.15.2...v1.16.0) - [github.com/psf/black: 24.8.0 → 24.10.0](psf/black@24.8.0...24.10.0) Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com> * Parametrize stack version and update app config (#54) * parametrize stack version * update data update date * set Google tag manager ID * rename `stack_version` to `image_version` * Increase GH workflow timeout (#55) A change[1] was made to update all containers at the same time which takes longer to deploy so we need to increase the deployment timeout. [1] #54 * Update stage environment (#57) * [pre-commit.ci] pre-commit autoupdate (#48) updates: - [github.com/PyCQA/flake8: 7.1.0 → 7.1.1](PyCQA/flake8@7.1.0...7.1.1) - [github.com/awslabs/cfn-python-lint: v1.9.0 → v1.15.0](aws-cloudformation/cfn-lint@v1.9.0...v1.15.0) - [github.com/psf/black: 24.4.2 → 24.8.0](psf/black@24.4.2...24.8.0) - [github.com/sirosen/check-jsonschema: 0.29.1 → 0.29.2](python-jsonschema/check-jsonschema@0.29.1...0.29.2) Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com> * [IT-3918] Fix the image URLs returned by the image service (#51) * update the config of the image service * use `{fully_qualified_domain_name}` * use an f-string * [pre-commit.ci] pre-commit autoupdate (#52) updates: - [github.com/awslabs/cfn-python-lint: v1.15.0 → v1.15.2](aws-cloudformation/cfn-lint@v1.15.0...v1.15.2) - [github.com/sirosen/check-jsonschema: 0.29.2 → 0.29.3](python-jsonschema/check-jsonschema@0.29.2...0.29.3) Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com> * [pre-commit.ci] pre-commit autoupdate (#53) updates: - [github.com/pre-commit/pre-commit-hooks: v4.6.0 → v5.0.0](pre-commit/pre-commit-hooks@v4.6.0...v5.0.0) - [github.com/awslabs/cfn-python-lint: v1.15.2 → v1.16.0](aws-cloudformation/cfn-lint@v1.15.2...v1.16.0) - [github.com/psf/black: 24.8.0 → 24.10.0](psf/black@24.8.0...24.10.0) Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com> * Parametrize stack version and update app config (#54) * parametrize stack version * update data update date * set Google tag manager ID * rename `stack_version` to `image_version` * Increase GH workflow timeout (#55) A change[1] was made to update all containers at the same time which takes longer to deploy so we need to increase the deployment timeout. [1] #54 --------- Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com> Co-authored-by: Khai Do <[email protected]> * [pre-commit.ci] pre-commit autoupdate (#56) updates: - [github.com/awslabs/cfn-python-lint: v1.16.0 → v1.16.1](aws-cloudformation/cfn-lint@v1.16.0...v1.16.1) - [github.com/sirosen/check-jsonschema: 0.29.3 → 0.29.4](python-jsonschema/check-jsonschema@0.29.3...0.29.4) Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com> * [pre-commit.ci] pre-commit autoupdate (#59) updates: - [github.com/awslabs/cfn-python-lint: v1.16.1 → v1.18.1](aws-cloudformation/cfn-lint@v1.16.1...v1.18.1) Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com> * [pre-commit.ci] pre-commit autoupdate (#60) updates: - [github.com/awslabs/cfn-python-lint: v1.18.1 → v1.18.2](aws-cloudformation/cfn-lint@v1.18.1...v1.18.2) Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com> * Set concurrency to 5 (best results) (#61) * [pre-commit.ci] pre-commit autoupdate (#62) updates: - [github.com/awslabs/cfn-python-lint: v1.18.2 → v1.18.4](aws-cloudformation/cfn-lint@v1.18.2...v1.18.4) Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com> * update image tag, app version and data release date (#63) * [pre-commit.ci] pre-commit autoupdate (#66) updates: - [github.com/awslabs/cfn-python-lint: v1.18.4 → v1.19.0](aws-cloudformation/cfn-lint@v1.18.4...v1.19.0) Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com> * Refactor mounting volumes (#67) The current implementation to mount volumes was very specific to one container. We are replacing it with an implementation that is much more generic to make it easy to mount volumes in other containers. * [IT-4003] Auto-update pre-commit hook versions monthly Change the frequency that PRs to update pre-commit hook versions are auto-generated from weekly (the default) to monthly. * Update to OC v1.1.1 (#69) * Update to v1.1.1 * update data updated on * [pre-commit.ci] pre-commit autoupdate (#70) updates: - [github.com/awslabs/cfn-python-lint: v1.19.0 → v1.20.1](aws-cloudformation/cfn-lint@v1.19.0...v1.20.1) - [github.com/sirosen/check-jsonschema: 0.29.4 → 0.30.0](python-jsonschema/check-jsonschema@0.29.4...0.30.0) Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com> * [IT-3951] Fix guardduty container (#71) We enable guardduty security monitoring for ECS in every account. For that to work we need to give Fragate tasks access to do ECS stuff with the service-role/AmazonECSTaskExecutionRolePolicy[1]. [1] https://docs.aws.amazon.com/guardduty/latest/ug/prereq-runtime-monitoring-ecs-support.html#before-enable-runtime-monitoring-ecs * remove source.bat (#74) * Add Docker in Docker to the dev container (#73) * Add Docker in Docker to the devcontainer * add docs about docker * forward local environment variables to the devcontainer * remove containerEnv * Add AWS Lambda for upcoming data integration (ARCH-356) (#72) * update docs on setup tools * define lambda role and function * update path to Dockerfile * update README * trigger the lambda every 5 minutes * use plural form of the unit * Remove lambda fct architecture * Migrate data integration code to L2 constructs * Add @DataClass to DataIntegrationProps * Add docstrings * Replace `_lambda` by `lambda_` * Add docstrings * Add docstrings * Externalize the description of the schedule (#75) * define lambda role and function * update path to Dockerfile * Externalize the description of the schedule * Merge dev to stage (#76) * [pre-commit.ci] pre-commit autoupdate (#48) updates: - [github.com/PyCQA/flake8: 7.1.0 → 7.1.1](PyCQA/flake8@7.1.0...7.1.1) - [github.com/awslabs/cfn-python-lint: v1.9.0 → v1.15.0](aws-cloudformation/cfn-lint@v1.9.0...v1.15.0) - [github.com/psf/black: 24.4.2 → 24.8.0](psf/black@24.4.2...24.8.0) - [github.com/sirosen/check-jsonschema: 0.29.1 → 0.29.2](python-jsonschema/check-jsonschema@0.29.1...0.29.2) Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com> * [IT-3918] Fix the image URLs returned by the image service (#51) * update the config of the image service * use `{fully_qualified_domain_name}` * use an f-string * [pre-commit.ci] pre-commit autoupdate (#52) updates: - [github.com/awslabs/cfn-python-lint: v1.15.0 → v1.15.2](aws-cloudformation/cfn-lint@v1.15.0...v1.15.2) - [github.com/sirosen/check-jsonschema: 0.29.2 → 0.29.3](python-jsonschema/check-jsonschema@0.29.2...0.29.3) Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com> * [pre-commit.ci] pre-commit autoupdate (#53) updates: - [github.com/pre-commit/pre-commit-hooks: v4.6.0 → v5.0.0](pre-commit/pre-commit-hooks@v4.6.0...v5.0.0) - [github.com/awslabs/cfn-python-lint: v1.15.2 → v1.16.0](aws-cloudformation/cfn-lint@v1.15.2...v1.16.0) - [github.com/psf/black: 24.8.0 → 24.10.0](psf/black@24.8.0...24.10.0) Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com> * Parametrize stack version and update app config (#54) * parametrize stack version * update data update date * set Google tag manager ID * rename `stack_version` to `image_version` * Increase GH workflow timeout (#55) A change[1] was made to update all containers at the same time which takes longer to deploy so we need to increase the deployment timeout. [1] #54 * [pre-commit.ci] pre-commit autoupdate (#56) updates: - [github.com/awslabs/cfn-python-lint: v1.16.0 → v1.16.1](aws-cloudformation/cfn-lint@v1.16.0...v1.16.1) - [github.com/sirosen/check-jsonschema: 0.29.3 → 0.29.4](python-jsonschema/check-jsonschema@0.29.3...0.29.4) Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com> * [pre-commit.ci] pre-commit autoupdate (#59) updates: - [github.com/awslabs/cfn-python-lint: v1.16.1 → v1.18.1](aws-cloudformation/cfn-lint@v1.16.1...v1.18.1) Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com> * [pre-commit.ci] pre-commit autoupdate (#60) updates: - [github.com/awslabs/cfn-python-lint: v1.18.1 → v1.18.2](aws-cloudformation/cfn-lint@v1.18.1...v1.18.2) Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com> * Set concurrency to 5 (best results) (#61) * [pre-commit.ci] pre-commit autoupdate (#62) updates: - [github.com/awslabs/cfn-python-lint: v1.18.2 → v1.18.4](aws-cloudformation/cfn-lint@v1.18.2...v1.18.4) Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com> * update image tag, app version and data release date (#63) * [pre-commit.ci] pre-commit autoupdate (#66) updates: - [github.com/awslabs/cfn-python-lint: v1.18.4 → v1.19.0](aws-cloudformation/cfn-lint@v1.18.4...v1.19.0) Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com> * Refactor mounting volumes (#67) The current implementation to mount volumes was very specific to one container. We are replacing it with an implementation that is much more generic to make it easy to mount volumes in other containers. * [IT-4003] Auto-update pre-commit hook versions monthly Change the frequency that PRs to update pre-commit hook versions are auto-generated from weekly (the default) to monthly. * Update to OC v1.1.1 (#69) * Update to v1.1.1 * update data updated on * [pre-commit.ci] pre-commit autoupdate (#70) updates: - [github.com/awslabs/cfn-python-lint: v1.19.0 → v1.20.1](aws-cloudformation/cfn-lint@v1.19.0...v1.20.1) - [github.com/sirosen/check-jsonschema: 0.29.4 → 0.30.0](python-jsonschema/check-jsonschema@0.29.4...0.30.0) Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com> * [IT-3951] Fix guardduty container (#71) We enable guardduty security monitoring for ECS in every account. For that to work we need to give Fragate tasks access to do ECS stuff with the service-role/AmazonECSTaskExecutionRolePolicy[1]. [1] https://docs.aws.amazon.com/guardduty/latest/ug/prereq-runtime-monitoring-ecs-support.html#before-enable-runtime-monitoring-ecs * remove source.bat (#74) * Add Docker in Docker to the dev container (#73) * Add Docker in Docker to the devcontainer * add docs about docker * forward local environment variables to the devcontainer * remove containerEnv * Add AWS Lambda for upcoming data integration (ARCH-356) (#72) * update docs on setup tools * define lambda role and function * update path to Dockerfile * update README * trigger the lambda every 5 minutes * use plural form of the unit * Remove lambda fct architecture * Migrate data integration code to L2 constructs * Add @DataClass to DataIntegrationProps * Add docstrings * Replace `_lambda` by `lambda_` * Add docstrings * Add docstrings * Externalize the description of the schedule (#75) * define lambda role and function * update path to Dockerfile * Externalize the description of the schedule --------- Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com> Co-authored-by: Khai Do <[email protected]> Co-authored-by: Joni Harker <[email protected]> Co-authored-by: Joni Harker <[email protected]> * update dev (#78) * Increase role duration to 90 minutes (#79) --------- Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com> Co-authored-by: Khai Do <[email protected]> Co-authored-by: Joni Harker <[email protected]> Co-authored-by: Joni Harker <[email protected]>
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
Closes https://sagebionetworks.jira.com/browse/ARCH-318
The current deployment time takes about 1h15. This led to deployment timeouts. One way we handled them is to increase the max timeout.
This PR enables the concurrent deployment of containers to accelerate the overall deployment time. Based on local experiment:
This PR sets the concurrency value to 5 because of the diminishing return observed.