Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Deploy duracloud 8 to production #25

Closed
wants to merge 36 commits into from
Closed
Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
36 commits
Select commit Hold shift + click to select a range
822ea07
DTS specific docs
mark-cooper Mar 7, 2023
a9671b0
Merge pull request #2 from dts-hosting/deployment-dts-docs
dbernstein Mar 8, 2023
b510d7b
Merge branch 'main' into deployments-sync-031323
mark-cooper Mar 13, 2023
1672793
Merge pull request #3 from dts-hosting/deployments-sync-031323
mark-cooper Mar 13, 2023
2730e10
Merge branch 'main' into deployments-sync-031723
mark-cooper Mar 17, 2023
8885d0f
Merge pull request #4 from dts-hosting/deployments-sync-031723
mark-cooper Mar 17, 2023
dd80730
Add overcommit for precommit fmt check
mark-cooper Mar 22, 2023
3b90ff6
Merge pull request #5 from dts-hosting/overcommit
mark-cooper Mar 22, 2023
2a53ec4
Use data region lookup to replace missing aws_region var (#8) (#6)
mark-cooper Apr 4, 2023
060ad62
Initial development environment setup (#7)
mark-cooper Apr 19, 2023
63f5947
Merge from main 2023-05-08 (#12)
dbernstein May 8, 2023
a9dd1dd
Deployments sync 2023 05 09 (#13)
dbernstein May 9, 2023
ae725ab
Update mill version to 4.2.0-SNAPSHOT for testing. (#10)
dbernstein May 10, 2023
62be5a6
update to latest snapshot version of duracloud. (#14)
dbernstein May 10, 2023
f4ca3c1
Initial setup for DuraCloud production environment (#9)
mark-cooper May 18, 2023
d6797a8
Update backends to use us-east-1 region (#15)
dbernstein May 18, 2023
7426233
update production duracloud zip (#16)
dbernstein May 18, 2023
947d694
Update to latest mill and mc versions. (#17)
dbernstein May 18, 2023
d5e3751
Deployments sync 2023 05 19 (#18)
dbernstein May 19, 2023
6af8b74
Deployments sync 20230602 (#19)
dbernstein Jun 2, 2023
bd4bbf8
Deployments sync 20230608 (#21)
dbernstein Jun 8, 2023
0d6dde6
Deployments sync 20230609 (#23)
dbernstein Jun 9, 2023
17b946f
Set development max worker counts to 2 (#24)
dbernstein Jun 9, 2023
36d7581
Use git merge for upstream sync (#25)
mark-cooper Aug 11, 2023
a3f1a7a
Deployments sync 2023 08 11 (#26)
dbernstein Aug 12, 2023
a28429b
Add updated TF Lock file. (#27)
dbernstein Aug 14, 2023
552042b
Dial down bit and dup workers to 1 from 2 in dev env. (#28)
dbernstein Aug 28, 2023
3db7dac
Deployments sync 2023 09 06 (#30)
dbernstein Sep 6, 2023
d1d5c85
Deployments sync 2023 09 06 2 (#31)
dbernstein Sep 6, 2023
20f4168
Deployments sync 2023 09 07 (#32)
dbernstein Sep 8, 2023
3a63fc0
Deployments sync 2023 09 11 (#33)
dbernstein Sep 11, 2023
afc77f0
Updates development and production mill to 4.2.0 (#34)
dbernstein Sep 12, 2023
38e580b
Deployments sync 2023 09 18 (#35)
dbernstein Sep 18, 2023
37e3ca0
Update the development environment with the latest from the developme…
dbernstein Oct 24, 2023
74235c3
Deployments sync 2023 10 26 (#37)
dbernstein Oct 26, 2023
ada26a0
Deploy duracloud 8, management console 8 and mill 5 to production.
dbernstein Nov 16, 2023
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
6 changes: 6 additions & 0 deletions .overcommit.yml
Original file line number Diff line number Diff line change
@@ -0,0 +1,6 @@
PreCommit:
Terraform:
enabled: true
command: ['terraform', 'fmt']
flags: ['-check', '-recursive']
install_command: tfenv install
1 change: 1 addition & 0 deletions .ruby-version
Original file line number Diff line number Diff line change
@@ -0,0 +1 @@
3.2.1
1 change: 1 addition & 0 deletions .terraform-version
Original file line number Diff line number Diff line change
@@ -0,0 +1 @@
1.3.5
103 changes: 103 additions & 0 deletions DTS.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,103 @@
# DTS

The [dts-hosting/duracloud-aws-deployer](https://github.com/dts-hosting/duracloud-aws-deployer)
repository is a downstream fork of the [duracloud/duracloud-aws-deployer](https://github.com/duracloud/duracloud-aws-deployer)
(upstream) project.

The dts deployer `main` branch is kept in a one-to-one relationship with the upstream
`main` branch. Upstream and DTS changes are integrated into the `deployments` branch,
which is where the various DuraCloud environments are defined and is the default branch
in GitHub (TODO).

## Repository setup

### Install local dependencies (optional)

First install:

- [Rbenv](https://github.com/rbenv/rbenv)
- [Tfenv](https://github.com/tfutils/tfenv)

Then run:

```bash
make install
```

### Sync with upstream

To work with the upstream DuraCloud deployer project:

```bash
git clone https://github.com/dts-hosting/duracloud-aws-deployer.git
git remote add upstream https://github.com/duracloud/duracloud-aws-deployer.git
```

Running `git remote -v` should show something like:

```txt
origin https://github.com/dts-hosting/duracloud-aws-deployer.git (fetch)
origin https://github.com/dts-hosting/duracloud-aws-deployer.git (push)
upstream https://github.com/duracloud/duracloud-aws-deployer.git (fetch)
upstream https://github.com/duracloud/duracloud-aws-deployer.git (push)
```

To incorporate upstream changes into the dts downstream fork:

```bash
git fetch --all
git checkout main
git merge upstream/main
git push origin main
```

The upstream and downstream main branches are now equivalent.

### Update the deployments branch from main

```bash
git fetch --all
git checkout deployments
git checkout -b deployments-sync-$date
git merge main
git push origin deployments-sync-$date
```

Make a PR to the `deployments` branch.

## AWS accounts / DuraCloud environments

For general considerations around AWS profile configurations refer to the
[infra](https://github.com/dts-hosting/infra/blob/main/docs/AWS.md) repository.

For DuraCloud AWS cli interactions define profiles for dev, prod & test:

```txt
[duraclouddev]
role_arn = arn:aws:iam::380144836391:role/OrganizationAccountAccessRole
source_profile = default
region = us-west-2

[duracloudprod]
role_arn = arn:aws:iam::863649442906:role/OrganizationAccountAccessRole
source_profile = default
region = us-east-1

[duracloudtest]
role_arn = arn:aws:iam::442366795148:role/OrganizationAccountAccessRole
source_profile = default
region = us-west-2
```

To use the profiles you'll generally need to add `--profile $profile` to
any AWS command examples provided in the upstream documentation.

For example, to check for a required parameter:

```bash
aws ssm get-parameter --name duracloud_artifact_bucket --profile duraclouddev
```

## Google reCAPTCHA

TODO: add details for this here.
12 changes: 12 additions & 0 deletions Makefile
Original file line number Diff line number Diff line change
@@ -0,0 +1,12 @@
SHELL:=/bin/bash

.PHONY: install
install: ## make install # Install dependencies
@rbenv install -s
@gem install overcommit && overcommit --install && overcommit --sign pre-commit
@tfenv install

help:
@grep -E '^[a-zA-Z_-]+:.*?## .*$$' $(MAKEFILE_LIST) | sort | awk 'BEGIN {FS = ":.*?## "}; {printf "\033[36m%-30s\033[0m %s\n", $$1, $$2}'

.DEFAULT_GOAL := help
47 changes: 27 additions & 20 deletions duracloud/main.tf
Original file line number Diff line number Diff line change
@@ -1,10 +1,12 @@
data "aws_region" "current" {}

module "common_parameters" {
source = "../modules/common_parameters"
}

locals {
cloud_init_props = {
aws_region = var.aws_region
aws_region = data.aws_region.current.name
}
}

Expand All @@ -19,7 +21,7 @@ resource "aws_s3_object" "duracloud_config_properties" {
}

data "aws_iam_instance_profile" "duracloud" {
name = "duracloud-instance-profile"
name = "${var.stack_name}-dc-ip"
}


Expand Down Expand Up @@ -143,7 +145,7 @@ data "aws_iam_role" "beanstalk_service" {
}

resource "aws_elastic_beanstalk_application" "duracloud" {
name = "DuraCloud"
name = "DuraCloud (${var.stack_name})"
description = "DuraCloud Beanstalk Application (${var.stack_name})"

appversion_lifecycle {
Expand All @@ -169,7 +171,7 @@ resource "aws_elastic_beanstalk_application_version" "default" {
resource "aws_elastic_beanstalk_configuration_template" "config" {
name = "duracloud-config"
application = aws_elastic_beanstalk_application.duracloud.name
solution_stack_name = "64bit Amazon Linux 2 v4.2.16 running Tomcat 8.5 Corretto 11"
solution_stack_name = var.solution_stack

setting {
namespace = "aws:ec2:vpc"
Expand Down Expand Up @@ -212,8 +214,13 @@ resource "aws_elastic_beanstalk_configuration_template" "config" {
name = "LoadBalancerType"
value = "application"
}
}

setting {
namespace = "aws:elasticbeanstalk:environment:process:default"
name = "StickinessEnabled"
value = "true"
}
}

resource "aws_elastic_beanstalk_environment" "duracloud" {
name = "${var.stack_name}-core"
Expand Down Expand Up @@ -245,22 +252,22 @@ resource "aws_elastic_beanstalk_environment" "duracloud" {
value = var.duracloud_instance_class
}

}
setting {
namespace = "aws:autoscaling:asg"
name = "MinSize"
value = var.minimum_instance_count
}

resource "aws_alb_target_group" "duracloud" {
name = "duracloud-alb-target-group"
target_type = "alb"
port = 80
protocol = "TCP"
vpc_id = data.aws_vpc.duracloud.id
stickiness {
type = "app_cookie"
cookie_name = "jsessionid"
setting {
namespace = "aws:autoscaling:asg"
name = "MaxSize"
value = var.maximum_instance_count
}
}

resource "aws_alb_target_group_attachment" "duracloud" {
target_group_arn = aws_alb_target_group.duracloud.arn
target_id = aws_elastic_beanstalk_environment.duracloud.load_balancers[0]
port = 80
setting {
namespace = "aws:elasticbeanstalk:environment:process:default"
name = "HealthCheckPath"
value = "/duradmin/login"
}
}

15 changes: 15 additions & 0 deletions duracloud/variables.tf
Original file line number Diff line number Diff line change
Expand Up @@ -8,6 +8,11 @@ variable "duracloud_zip" {

}

variable "solution_stack" {
description = "The AWS Solution Stack to use with the Elastic Beanstalk."
default = "64bit Amazon Linux 2 v4.3.7 running Tomcat 8.5 Corretto 11"
}

variable "duracloud_instance_class" {
description = "The instance size of worker ec2 instance class"
default = "m5.large"
Expand All @@ -20,3 +25,13 @@ variable "ec2_keypair" {
variable "stack_name" {
description = "The name of the duracloud stack."
}

variable "minimum_instance_count" {
description = "The minimum number of instances to run"
default = 2
}

variable "maximum_instance_count" {
description = "The minimum number of instances to run"
default = 6
}
1 change: 1 addition & 0 deletions env/development/duracloud/.terraform-version
25 changes: 25 additions & 0 deletions env/development/duracloud/.terraform.lock.hcl

Some generated files are not rendered by default. Learn more about how customized files appear on GitHub.

39 changes: 39 additions & 0 deletions env/development/duracloud/backend.tf
Original file line number Diff line number Diff line change
@@ -0,0 +1,39 @@
# TERRAFORM BACKEND & PROVIDER CFG
terraform {
required_version = "1.3.5"

cloud {
organization = "Lyrasis"
workspaces {
name = "duracloud-development-duracloud"
}
}

required_providers {
aws = {
source = "hashicorp/aws"
version = "4.61.0"
}
}
}

provider "aws" {
region = "us-west-2"
allowed_account_ids = ["380144836391"]

assume_role {
role_arn = "arn:aws:iam::380144836391:role/OrganizationAccountAccessRole"
session_name = "duracloud-development-duracloud"
external_id = "duracloud-development-duracloud"
}

default_tags {
tags = {
Service = "duracloud"
Department = "dts"
Environment = "development"
Project = "duracloud"
Terraform = true
}
}
}
1 change: 1 addition & 0 deletions env/development/duracloud/duracloud.auto.tfvars
Original file line number Diff line number Diff line change
@@ -0,0 +1 @@
duracloud_zip = "duracloud-beanstalk-v7.2.0-SNAPSHOT-25f45e8.zip"
1 change: 1 addition & 0 deletions env/development/duracloud/main.tf
1 change: 1 addition & 0 deletions env/development/duracloud/resources
1 change: 1 addition & 0 deletions env/development/duracloud/shared.auto.tfvars
1 change: 1 addition & 0 deletions env/development/duracloud/variables.tf
1 change: 1 addition & 0 deletions env/development/management-console/.terraform-version
25 changes: 25 additions & 0 deletions env/development/management-console/.terraform.lock.hcl

Some generated files are not rendered by default. Learn more about how customized files appear on GitHub.

Loading