Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Init access the data #44

Merged
merged 4 commits into from
Jan 24, 2024
Merged

Init access the data #44

merged 4 commits into from
Jan 24, 2024

Conversation

nyarly
Copy link
Member

@nyarly nyarly commented Dec 8, 2023

This is as embarrassingly huge PR. Sorry for that.

Contained within are the additions and changes I needed to make in order to get Access the Data ready to deploy. Note well! AtD isn't successfully deployed yet, but that has to do with issues with CKAN, not the incubator platform.

Introduction

(yeah, it's that kind of PR)

As a rough guide to what's changed:
The headline is the multi-container-service module. It's designed with AtD in mind, but in general I suspect that being able to deploy e.g. a web backend and Redis (say) will be really valuable to Hack for LA. MCS also covers details like setting up public and private DNS. MCS covers a couple other outstanding Incubator issues, c.f. #18

MCS also drives an ecr module, to create container registries that developers can push to to update their services.

Other new modules include database which uses the cyrilgdn/postgresql provider to provision postgres databases and roles within the shared RDS instance. All credentials are stored in SSM parameters as SecureStrings, so there's no toxic secrets to manage in IaC. (c.f. #1)

Because that's such a useful pattern, there's also a cheap-secrets module, which allows for purpose-agnostic secrets to be securely generated and stashed in SSM. This isn't 100% ideal, but the benefit is that managing secrets this way is free, as compared to using AWS SecretStore. The downside is that the secret material might be stored in terraform state files and would theoretically be extractable. The same personnel who could do that, though, could simply query SSM, or deploy a task that would forward secrets to them.

The Terraform shared_resources modules have been fleshed out significantly, such that we can get data from all of them.

Overall, I feel very good about these changes, and would recommend that future Incubator projects be built using multi-container-service on AtD's pattern even if they only deploy a single container.

@@ -0,0 +1,42 @@
// Get configuration from the shared infrastructure
Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Projects wanting to copy from AtD should be able to copy this file verbatim. There's some boilerplate that should be the same for every project.

@@ -0,0 +1,148 @@
locals {
Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The contents of this file were designed to contain exactly what varies between projects. Other projects might crib from this, but expect to change basically every value.

@@ -26,31 +26,72 @@ data "terraform_remote_state" "shared" {
}
}

locals {
configuration = {
Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Prefer always to use the single shared configuration object - it reduces the coupling between modules, who only need to name the fields that they care about.

Each field is exported as an output separately, to support legacy use.

@@ -0,0 +1,89 @@
terraform {
Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Ideally, we would migrate to this to the complete exclusion of the DB lambda

@@ -0,0 +1,208 @@
locals {
Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This was copied from the existing ecs module and modified for use in MCS.

health_check_path = each.value.health_check_path
log_group = aws_cloudwatch_log_group.cwlogs.name
container_port = each.value.port
container_env_vars = merge(each.value.env_vars, {
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@nyarly I see a few formatting things like this w/ alignment...maybe run a final tf fmt?

Copy link
Member

@chelseybeck chelseybeck left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Great work, @nyarly!

@nyarly nyarly merged commit dbc0981 into hackforla:main Jan 24, 2024
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

2 participants