initial commit for filmdrop deployment of titiler and console-ui (#1)

* initial commit for filmdrop deployment of titiler and console-ui

.github/pull_request_template.md

@@ -0,0 +1,13 @@
+**Related Issue(s):** 
+
+- #
+
+
+**Proposed Changes:**
+
+1. 
+2. 
+
+**PR Checklist:**
+
+- [ ] I have added my changes to the CHANGELOG **or** a CHANGELOG entry is not required.

.github/workflows/ci.yaml

@@ -0,0 +1,121 @@
+name: Release CI
+
+env:
+  PROJECT_NAME: mmw
+  FILMDROP_TERRAFORM_RELEASE: v2.23.0 # keep this up to date!
+  STAGE: staging
+  CI: true
+
+on:
+  push:
+#    branches: ["main"]
+    tags: ["v*.*.*"]
+
+jobs:
+  deploy:
+    permissions:
+      id-token: write
+      contents: read
+    runs-on: ubuntu-latest
+    steps:
+      - uses: hashicorp/setup-terraform@v3
+        with:
+          terraform_version: "1.7.5"
+
+      - name: Preparing Environment
+        id: prep_env
+        run: |
+          echo "Creating terraform backend file ..."
+          echo '' > config.s3.backend.tf
+          echo 'terraform {' >> config.s3.backend.tf
+          echo '  backend "s3" {' >> config.s3.backend.tf
+          echo '    encrypt = true' >> config.s3.backend.tf
+          echo "    bucket = \"${{ secrets.TF_STATE_BUCKET }}\"" >> config.s3.backend.tf
+          echo "    dynamodb_table = \"${{ secrets.TF_STATE_LOCK_TABLE }}\"" >> config.s3.backend.tf
+          echo "    key = \"${PROJECT_NAME}-${STAGE}.tfstate\"" >> config.s3.backend.tf
+          echo "    region = \"${{ secrets.AWS_REGION }}\"" >> config.s3.backend.tf
+          echo '  }' >> config.s3.backend.tf
+          echo '}' >> config.s3.backend.tf
+          cat config.s3.backend.tf
+          echo "Using FilmDrop Terraform ${FILMDROP_TERRAFORM_RELEASE} release..."
+          wget -q -O filmdrop-aws-tf-modules-${FILMDROP_TERRAFORM_RELEASE:1}.tgz https://github.com/Element84/filmdrop-aws-tf-modules/archive/refs/tags/${FILMDROP_TERRAFORM_RELEASE}.tar.gz
+          tar -xzf filmdrop-aws-tf-modules-${FILMDROP_TERRAFORM_RELEASE:1}.tgz
+          mv filmdrop-aws-tf-modules-${FILMDROP_TERRAFORM_RELEASE:1}/* .
+
+      - name: Configure Terraform Init Credentials
+        id: init_creds
+        uses: aws-actions/configure-aws-credentials@v4
+        with:
+          aws-region: ${{ secrets.AWS_REGION }}
+          role-to-assume: ${{ secrets.AWS_ROLE }}
+          role-session-name: GitHubReleaseInit
+
+      - name: Terraform Init
+        id: tf_init
+        run: terraform init
+
+      - name: Terraform Validate
+        id: tf_validate
+        run: terraform validate
+
+      - name: Configure Terraform Plan Credentials
+        id: plan_creds
+        uses: aws-actions/configure-aws-credentials@v4
+        with:
+          aws-region: ${{ secrets.AWS_REGION }}
+          role-to-assume: ${{ secrets.AWS_ROLE }}
+          role-session-name: GitHubReleasePlan
+
+      - name: Terraform Plan
+        id: tf_plan
+        run: terraform plan -var-file="${STAGE}.tfvars" -out ${STAGE}.tfplan -lock=false
+
+      - name: Configure Terraform Apply Credentials
+        id: apply_creds
+        uses: aws-actions/configure-aws-credentials@v4
+        with:
+          aws-region: ${{ secrets.AWS_REGION }}
+          role-to-assume: ${{ secrets.AWS_ROLE }}
+          role-session-name: GitHubReleaseApply
+
+      - run: terraform apply -lock=false -input=false ${STAGE}.tfplan
+
+      - name: Post status to Slack channel
+        id: tf_apply_successs
+        if: steps.tf_apply.outcome == 'success'
+        continue-on-error: true
+        uses: slackapi/[email protected]
+        with:
+          channel-id: ${{ secrets.SLACK_CHANNEL_ID }}
+          slack-message: ":badger_dance: ${PROJECT_NAME}-${STAGE}-titiler ${{  github.ref_name }} terraform apply job has succeeded!\n${{ github.event.pull_request.html_url || github.event.head_commit.url }}"
+        env:
+          SLACK_BOT_TOKEN: ${{ secrets.SLACK_BOT_TOKEN }}
+
+      - name: Post status to Slack channel
+        id: tf_apply_failure
+        if: steps.tf_apply.outcome != 'success'
+        continue-on-error: true
+        uses: slackapi/[email protected]
+        with:
+          channel-id: ${{ secrets.SLACK_CHANNEL_ID }}
+          slack-message: ":sadpanda: ${PROJECT_NAME}-${STAGE}-titiler ${{  github.ref_name }} terraform apply has failed!\n:alert: make sure cleanup job deletes all AWS resources!\n${{ github.event.pull_request.html_url || github.event.head_commit.url }}"
+        env:
+          SLACK_BOT_TOKEN: ${{ secrets.SLACK_BOT_TOKEN }}
+
+  pre-commit:
+    runs-on: ubuntu-latest
+    steps:
+      - uses: actions/checkout@v4
+      - uses: hashicorp/setup-terraform@v3
+        with:
+          terraform_version: "1.7.5"
+      - uses: terraform-linters/setup-tflint@v4
+        with:
+          tflint_version: "v0.51.1"
+      - uses: actions/setup-python@v5
+        with:
+          python-version: "3.11"
+      - name: Install pre-commit
+        run: pip install pre-commit
+      - name: Run pre-commit
+        run: pre-commit run --all-files

.gitignore

@@ -0,0 +1,26 @@
+.terraform/
+modules
+profiles
+tfplan
+*.tf
+terraform-visual-report
+plan.json
+saved.plan
+*.tfstate
+*.tfstate.*
+.terraform.lock.hcl
+terraform.tgz
+bin/terraform
+*.auto.tfvars
+*.swp
+*.swo
+*.tmp
+*.orig
+*.out
+*.zip
+*output
+node_modules
+package-lock.json
+plan.out
+.pytest*
+__pycache__

.markdownlint.json

@@ -0,0 +1,3 @@
+{
+    "MD024": false
+}

.pre-commit-config.yaml

@@ -0,0 +1,12 @@
+repos:
+  - repo: https://github.com/antonbabenko/pre-commit-terraform
+    rev: v1.83.2
+    hooks:
+      - id: terraform_fmt
+      - id: terraform_tflint
+      - id: terraform_validate
+      # - id: terraform_tfsec
+      # - id: terraform_docs
+      # - id: terraform_checkov
+      # - id: terrascan
+      # - id: tfupdate

CHANGELOG.md

@@ -0,0 +1,11 @@
+# Changelog
+
+All notable changes to this project will be documented in this file.
+
+The format is based on [Keep a Changelog](http://keepachangelog.com/en/1.0.0/).
+
+## unreleased
+
+### Added
+
+- Initial release for staging

README.md

@@ -1 +1,90 @@
+<!-- omit from toc -->
 # Model My Watershed Tiler Deployment
+
+- [Setup](#setup)
+- [Deploying](#deploying)
+  - [Via GitHub Actions](#via-github-actions)
+  - [Manual Deploy](#manual-deploy)
+- [Destroy](#destroy)
+  - [Via GitHub Actions](#via-github-actions-1)
+  - [Manual Deploy](#manual-deploy-1)
+
+This repository deploys [TiTiler](https://github.com/developmentseed/titiler)
+(as the fork [titiler-mosaicjson](https://github.com/Element84/titiler-mosaicjson))
+and [FilmDrop UI](https://github.com/Element84/filmdrop-ui) via the
+[FilmDrop Infrastructure Terraform Modules](https://github.com/Element84/filmdrop-aws-tf-modules).
+
+## Setup
+
+1. In the AWS Accounts to be deployed into, create the bootstrap resources as
+   outlined in <bootstrap/README.md>.
+
+## Deploying
+
+### Via GitHub Actions
+
+Create a GitHub Environment (e.g., `staging`) with these Environment Secrets:
+
+| Variable              | Description                                             | Example                                                                               |
+| --------------------- | ------------------------------------------------------- | ------------------------------------------------------------------------------------- |
+| `AWS_REGION`          | region to deploy into                                   | `us-west-2`                                                                           |
+| `AWS_ROLE`            | the ARN of the AWS Role to use for deploy               | `arn:aws:iam::0123456789:role/appFilmDropDeployRoleBootstrap-DeployRole-Wfx5HwlneOVM` |
+| `TF_STATE_BUCKET`     | the bucket use for storing Terraform state              | `filmdrop-{project_name}-{region}-terraform-state-{random_string}`                    |
+| `TF_STATE_LOCK_TABLE` | the DynamoDB table to use for Terraform locks           | `filmdrop-terraform-state-locks`                                                      |
+| `SLACK_CHANNEL_ID`    | ID of Slack channel to post deploy status notifications | `D26F29X7OB3`                                                                         |
+| `SLACK_BOT_TOKEN`     | Slack Bot Token                                         | alphanumeric string                                                                   |
+
+TODO: finish
+
+### Manual Deploy
+
+By default, Terraform will use a local store for state. If you want to configure
+this to use S3 and DynamoDB instead, in the same way the GitHub Actions build does,
+create a file to define the backend named `config.s3.backend.tf` with contents like:
+
+```text
+terraform {
+  backend "s3" {
+    encrypt        = true
+    bucket         = REPLACE_ME # with the bootstrapped bucket name
+    dynamodb_table = "filmdrop-terraform-state-locks"
+    key            = "mmw-{username}-test.tfstate" # replace with username
+    region         = "us-west-2"
+  }
+}
+```
+
+The `bucket` name will be the value to be set for `TF_STATE_BUCKET`, e.g.,
+`filmdrop-{project_name}-{region}-terraform-state-{random_string}`.
+
+Download the filmdrop-aws-tf-modules source:
+
+```shell
+FILMDROP_TERRAFORM_RELEASE=v2.23.0 ./scripts/retrieve_tf_modules.sh
+```
+
+Re-run this anytime you with to uptake a new `filmdrop-aws-tf-modules` release,
+in addition to updating the env var in `.github/workflows/ci.yaml`.
+
+Run the Terraform commands to initialize, validate, plan, and apply the
+configuration:
+
+```shell
+terraform init
+terraform validate
+terraform plan -var-file=staging.tfvars -out tfplan
+terraform apply -input=false tfplan
+```
+
+If you prefer to use a local state file, can delete the `config.s3.backend.tf`
+file and run `terraform init` again without it.
+
+## Destroy
+
+### Via GitHub Actions
+
+TODO
+
+### Manual Deploy
+
+Run `terraform destroy -var-file=staging.tfvars -input=false`

bootstrap/README.md

@@ -0,0 +1,70 @@
+# Bootstrap
+
+- [Bootstrap](#bootstrap)
+  - [Deployment Role](#deployment-role)
+  - [Terraform State](#terraform-state)
+    - [Input Parameters](#input-parameters)
+    - [Deploy](#deploy)
+
+This project includes infrastructure resources that must be deployed separately
+and prior to any FilmDrop resources.
+
+## Deployment Role
+
+The deployment Role is an IAM Role that is assumed when deploying all FilmDrop infrastructure.
+
+The deployment Role should be created once per AWS Account.
+
+You can deploy this role through CloudFormation
+with the following command:
+
+```shell
+aws cloudformation deploy \
+  --stack-name "appFilmDropDeployRoleBootstrap" \
+  --template-file bootstrap/deploy_role_cfn.yml \
+  --capabilities=CAPABILITY_NAMED_IAM
+```
+
+## Terraform State
+
+FilmDrop resources are deployed as [Terraform](https://www.terraform.io/) modules
+which require a managed state file as part of the deployment. This project
+bootstraps the resources necessary to support the Terraform state file and
+backend configuration.
+
+The Terraform state S3 Bucket and DynamoDB Table should be created once per AWS
+Account + Region.
+
+### Input Parameters
+
+Input parameters for the Terraform state file resources are found in
+[terraform_state_params.json](./terraform_state_params.json) and should be
+modified for your project.
+
+The Terraform state S3 Bucket name must be globally unique - it's recommended to
+use a combination or your AWS account alias, deploy region, and a random suffix
+in order to ensure uniqueness. For example:
+
+```text
+filmdrop-myProjectAccount-us-west-2-terraform-state-96842
+```
+
+The DynamoDB locks table name must be a unique name per AWS account and region.
+The table name defaults to `filmdrop-terraform-state-locks`, but can be overridden.
+
+### Deploy
+
+You can deploy these resources through CloudFormation with the following command,
+substituting the value of `TerraformStateBucketName`.
+
+```shell
+aws cloudformation deploy \
+  --stack-name "appFilmDropTerraformStateBootstrap" \
+  --template-file bootstrap/terraform_state_cfn.yml \
+  --parameter-overrides \
+    TerraformStateBucketName=REPLACEME
+```
+
+Once deployed you'll need the S3 Bucket name and DynamoDB table name to configure
+the Terraform backend. You'll add those values to the FilmDrop infrastructure
+project later.