Add ability to generate secrets for the application (#602)

⚠️ breaking change (change to way secrets are defined in app-config in
environment-variables.tf).
secrets are now defined as a map:

```
secrets = {
  ENV_VAR_NAME = {
    manage_method     = "code" or "manual"
    secret_store_path = "/ssm/param/name"
  }
}
```

It was previously defined as a list:

```
secrets = [
  {
    name = "ENV_VAR_NAME",
    ssm_param_name = "/ssm/param/name"
  }
]
```

* Add new module modules/secret for generating new secrets or
referencing existing secrets
* Refactor interface

To migrate:

* In app-config's environment-variables.tf, update secret definitions to
use the new format.
* For secrets managed outside of the project's codebase, set
manage_method = "manual"
* For secrets created within the project's codebase but defined
elsewhere, move (using [terraform
mv](https://developer.hashicorp.com/terraform/cli/commands/state/mv))
the aws_ssm_parameter to
module.secret[ENV_VAR_NAME].aws_ssm_parameter.secret

app/Dockerfile

@@ -1,4 +1,7 @@
-FROM python:3-alpine as release
+# Pin to Alpine 3.19 since aws-cli was removed in Alpine 3.20
+# see https://github.com/alpinelinux/docker-alpine/issues/396
+# https://wiki.alpinelinux.org/wiki/Release_Notes_for_Alpine_3.20.0#aws-cli
+FROM python:3-alpine3.19 as release
 
 RUN adduser --system --disabled-password --no-create-home app
 

app/app.py

@@ -75,7 +75,8 @@ def document_upload():
 @app.route("/secrets")
 def secrets():
     secret_sauce = os.environ["SECRET_SAUCE"]
-    return f'The secret sauce is "{secret_sauce}"'
+    random_secret = os.environ["RANDOM_SECRET"]
+    return f'The secret sauce is "{secret_sauce}".<br> The random secret is "{random_secret}".'
 
 
 @app.cli.command("etl", help="Run ETL job")

docs/infra/environment-variables-and-secrets.md

@@ -40,19 +40,23 @@ module "dev_config" {
 
 Secrets are a specific category of environment variables that need to be handled sensitively. Examples of secrets are authentication credentials such as API keys for external services. Secrets first need to be stored in AWS SSM Parameter Store as a `SecureString`. This section then describes how to make those secrets accessible to the ECS task as environment variables through the `secrets` configuration in the container definition (see AWS documentation on [retrieving Secrets Manager secrets through environment variables](https://docs.aws.amazon.com/AmazonECS/latest/developerguide/secrets-envvar-secrets-manager.html)).
 
-Secrets are defined in the same file that non-sensitive environment variables are defined, in the `app-config` module in the [environment-variables.tf file](/infra/app/app-config/env-config/environment-variables.tf). Modify the `secrets` list to define the secrets that the application will have access to. For each secret, `name` defines the environment variable name, and `ssm_param_name` defines the SSM parameter name that stores the secret value. For example:
+Secrets are defined in the same file that non-sensitive environment variables are defined, in the `app-config` module in the [environment-variables.tf file](/infra/app/app-config/env-config/environment-variables.tf). Modify the `secrets` map to define the secrets that the application will have access to. For each secret, the map key defines the environment variable name. The `managed_by` property, which can be set to `"generated"` or `"manual"`, defines whether or not to generate a random secret or to reference an existing secret that was manually created and stored into AWS SSM. The `secret_store_name` property defines the SSM parameter name that stores the secret value. If `managed_by = "generated"`, then `secret_store_name` is where terraform will store the secret. If `managed_by = "manual"`, then `secret_store_name` is where terraform will look for the existing secret. For example:
 
 ```terraform
 # environment-variables.tf
 
 locals {
-  secrets = [
-    {
-      name           = "SOME_API_KEY"
-      ssm_param_name = "/${var.app_name}-${var.environment}/secret-sauce"
+  secrets = {
+    GENERATED_SECRET = {
+      manage_method     = "generated"
+      secret_store_name = "/${var.app_name}-${var.environment}/generated-secret"
     }
-  ]
+    MANUALLY_CREATED_SECRET = {
+      manage_method     = "manual"
+      secret_store_name = "/${var.app_name}-${var.environment}/manually-created-secret"
+    }
+  }
 }
 ```
 
-> ⚠️ Make sure you store the secret in SSM Parameter Store before you try to add secrets to your application service, or else the service won't be able to start since the ECS Task Executor won't be able to fetch the configured secret.
+> ⚠️ For secrets with `managed_by = "manual"`, make sure you store the secret in SSM Parameter Store *before* you try to add configure your application service with the secrets, or else the service won't be able to start since the ECS Task Executor won't be able to fetch the configured secret.

infra/app/app-config/env-config/environment-variables.tf

@@ -12,12 +12,23 @@ locals {
   # Configuration for secrets
   # List of configurations for defining environment variables that pull from SSM parameter
   # store. Configurations are of the format
-  # { name = "ENV_VAR_NAME", ssm_param_name = "/ssm/param/name" }
-  secrets = [
-    # Example secret
-    # {
-    #   name           = "SECRET_SAUCE"
-    #   ssm_param_name = "/${var.app_name}-${var.environment}/secret-sauce"
+  # {
+  #   ENV_VAR_NAME = {
+  #     manage_method     = "generated" # or "manual" for a secret that was created and stored in SSM manually
+  #     secret_store_name = "/ssm/param/name"
+  #   }
+  # }
+  secrets = {
+    # Example generated secret
+    # RANDOM_SECRET = {
+    #   manage_method     = "generated"
+    #   secret_store_name = "/${var.app_name}-${var.environment}/random-secret"
     # }
-  ]
+
+    # Example secret that references a manually created secret
+    # SECRET_SAUCE = {
+    #   manage_method     = "manual"
+    #   secret_store_name = "/${var.app_name}-${var.environment}/secret-sauce"
+    # }
+  }
 }

infra/app/app-config/env-config/outputs.tf

@@ -30,7 +30,7 @@ output "service_config" {
       var.service_override_extra_environment_variables
     )
 
-    secrets = toset(local.secrets)
+    secrets = local.secrets
 
     file_upload_jobs = {
       for job_name, job_config in local.file_upload_jobs :

infra/app/service/main.tf

@@ -155,7 +155,12 @@ module "service" {
     BUCKET_NAME           = local.storage_config.bucket_name
   }, local.service_config.extra_environment_variables)
 
-  secrets = local.service_config.secrets
+  secrets = [
+    for secret_name in keys(local.service_config.secrets) : {
+      name      = secret_name
+      valueFrom = module.secrets[secret_name].secret_arn
+    }
+  ]
 
   extra_policies = {
     feature_flags_access = module.feature_flags.access_policy_arn,

infra/app/service/secrets.tf

@@ -0,0 +1,16 @@
+module "secrets" {
+  for_each = local.service_config.secrets
+
+  source = "../../modules/secret"
+
+  # When generating secrets and storing them in parameter store, append the
+  # terraform workspace to the secret store path if the environment is temporary
+  # to avoid conflicts with existing environments.
+  # Don't do this for secrets that are managed manually since the temporary
+  # environments will need to share those secrets.
+  secret_store_name = (each.value.manage_method == "generated" && local.is_temporary ?
+    "${each.value.secret_store_name}/${terraform.workspace}" :
+    each.value.secret_store_name
+  )
+  manage_method = each.value.manage_method
+}

infra/modules/secret/main.tf

@@ -0,0 +1,26 @@
+locals {
+  secret             = var.manage_method == "generated" ? aws_ssm_parameter.secret[0] : data.aws_ssm_parameter.secret[0]
+  access_policy_name = "${trimprefix(replace(local.secret.name, "/", "-"), "/")}-access"
+}
+
+resource "random_password" "secret" {
+  count = var.manage_method == "generated" ? 1 : 0
+
+  length           = 64
+  special          = true
+  override_special = "!#$%&*()-_=+[]{}<>:?"
+}
+
+resource "aws_ssm_parameter" "secret" {
+  count = var.manage_method == "generated" ? 1 : 0
+
+  name  = var.secret_store_name
+  type  = "SecureString"
+  value = random_password.secret[0].result
+}
+
+data "aws_ssm_parameter" "secret" {
+  count = var.manage_method == "manual" ? 1 : 0
+
+  name = var.secret_store_name
+}

infra/modules/secret/outputs.tf

@@ -0,0 +1,3 @@
+output "secret_arn" {
+  value = local.secret.arn
+}

infra/modules/secret/variables.tf

@@ -0,0 +1,22 @@
+variable "manage_method" {
+  type        = string
+  description = <<EOT
+    Method to manage the secret. Options are 'manual' or 'generated'.
+    Set to 'generated' to generate a random secret.
+    Set to 'manual' to reference a secret that was manually created and stored in AWS parameter store.
+    Defaults to 'generated'."
+    EOT
+  default     = "generated"
+  validation {
+    condition     = can(regex("^(manual|generated)$", var.manage_method))
+    error_message = "Invalid manage_method. Must be 'manual' or 'generated'."
+  }
+}
+
+variable "secret_store_name" {
+  type        = string
+  description = <<EOT
+    If manage_method is 'generated', path to store the secret in AWS parameter store.
+    If manage_method is 'manual', path to reference the secret in AWS parameter store.
+    EOT
+}

infra/modules/service/access-control.tf

@@ -68,7 +68,7 @@ data "aws_iam_policy_document" "task_executor" {
     content {
       sid       = "SecretsAccess"
       actions   = ["ssm:GetParameters"]
-      resources = local.secret_arn_patterns
+      resources = [for secret in var.secrets : secret.valueFrom]
     }
   }
 }

infra/modules/service/main.tf

@@ -94,7 +94,7 @@ resource "aws_ecs_task_definition" "app" {
         ]
       },
       environment = local.environment_variables,
-      secrets     = local.secrets,
+      secrets     = var.secrets,
       portMappings = [
         {
           containerPort = var.container_port,

infra/modules/service/variables.tf

@@ -92,8 +92,8 @@ variable "extra_environment_variables" {
 
 variable "secrets" {
   type = set(object({
-    name           = string
-    ssm_param_name = string
+    name      = string
+    valueFrom = string
   }))
   description = "List of configurations for defining environment variables that pull from SSM parameter store"
   default     = []