Implement CircleCI deployment when pushing to develop

[leothomas]

What I did:

Implemented CircleCI deployments to a staging CDK stack off of pushes to deployment. There are 2 main reasons for this feature:

• A staging backend stack allows us to more easily share "in-progress" features with partners (without having to set them up with a local dev environment)
• A staging backend stack allows us to test updated deployment procedures and any other aspects of the backend that may act differently between a local environment and the AWS environment (networking, rate limiting, etc). In this case, I enabled lambda deployment package generation through the CDK script (rather than docker commands directly in CircleCI - PR here (closed)), and I want to ensure the deployment goes smoothly by deploying from develop, before merging/deploying to prod.

How I did it:

I added a new step to the deployment script which deploys to a staging CDK stack when pushing to develop.

The lambda deployment package generation had originally been disabled in the CDK script and was being done directly CircleCI, due to the fact that lambda deployment package generation in CDK uses docker and it is not possible to mount docker container volumes in CircleCI, since CircleCI also runs in docker, and according to @CiaranEvans:

Running docker in docker? You're gonna have a bad time

In order to enable the CDK script to use docker when deploying the stack from within CircleCI I switched from a docker image executor to a machine executor, which runs the CI pipeline in a Linux VM rather than a docker container. This gives me full controls of the underlying file system, and allows me to mount the volumes needed when generating the lambda deployment package.

How you can test it:

Merge this PR to develop and watch a new staging stack get created in the NASA Impact AWS account!

Note:
Currently the staging stack fails to create because of a limit of 5 VPC's per region in AWS. us-east-1 has one for the prod stack, one for a stack I stood up to share contrail detections with the dev's at Impact, one for a test stack that used to be used by ESA devs (unclear wether it is still being used or not, I didn't want to delete it, just in case). There are 2 VPC's that don't belong to any Cloudformation stack.

Possible solutions:

• Delete the test stack if we can confirm it's no longer in use by the folks at ESA
• Figure out what those two other VPC's are in us-east-1 and delete one if it is not needed
• Deploy the staging stack to a different region
• Request a VPC-per-region limit increase from AWS
• Update the CDK stack to create a VPC if one doesn't already exist, but use an existing one if it exsits.

[ciaransweet]

Looks good to me, I think you've just left some debugging statements in... maybe?

I appreciate the direct quote 🤣

On the VPC topic, if one of those VPCs does what you need, you could just set your deployment to go into one of those for now, which is a quick fix!