Skip to content

Commit

Permalink
chore: 4-projects uat feedback issues (terraform-google-modules#459)
Browse files Browse the repository at this point in the history
  • Loading branch information
daniel-cit authored Apr 26, 2021
1 parent 1d3fbf8 commit 9bdf88f
Show file tree
Hide file tree
Showing 13 changed files with 52 additions and 32 deletions.
27 changes: 20 additions & 7 deletions 4-projects/README.md
Original file line number Diff line number Diff line change
Expand Up @@ -24,7 +24,7 @@ organizational policy.</td>
</tr>
<tr>
<td><a
href="https://github.com/terraform-google-modules/terraform-example-foundation/tree/master/2-environments">2-environments</a></td>
href="https://github.com/terraform-google-modules/terraform-example-foundation/tree/master/2-environments"><span style="white-space: nowrap;">2-environments</span></a></td>
<td>Sets up development, non-production, and production environments within the
Google Cloud organization that you've created.</td>
</tr>
Expand All @@ -41,6 +41,11 @@ up the global DNS hub.</td>
<td>Sets up a folder structure, projects, and application infrastructure pipeline for applications,
which are connected as service projects to the shared VPC created in the previous stage.</td>
</tr>
<tr>
<td><a
href="https://github.com/terraform-google-modules/terraform-example-foundation/tree/master/5-app-infra">5-app-infra</a></td>
<td>Deploy a simple <a href="https://cloud.google.com/compute/">Compute Engine</a> instance in one of the business unit projects using the infra pipeline set up in 4-projects.</td>
</tr>
</tbody>
</table>

Expand All @@ -49,7 +54,11 @@ For an overview of the architecture and the parts, see the

## Purpose

The purpose of this step is to set up the folder structure, projects, and infrastructure pipelines for applications that are connected as service projects to the shared VPC created in the previous stage. For each business unit, a shared `infra-pipeline` project is created along with Cloud Build triggers, CSRs for application infrastructure code and Google Cloud Storage buckets for state storage. This step follows the same [conventions](https://github.com/terraform-google-modules/terraform-example-foundation#branching-strategy) as the foundation pipeline deployed in [0-bootstrap](https://github.com/terraform-google-modules/terraform-example-foundation/blob/master/0-bootstrap/README.md). The Cloud Build SA used by this pipeline can impersonate the project SA by enabling the `enable_cloudbuild_deploy` flag and necessary roles can be granted to this SA via `sa_roles` as shown in this [example](business_unit_1/development/example_base_shared_vpc_project.tf). This pipeline can be utilized for deploying resources in projects across development/non-production/production with granular permissions.
The purpose of this step is to set up the folder structure, projects, and infrastructure pipelines for applications that are connected as service projects to the shared VPC created in the previous stage.
For each business unit, a shared `infra-pipeline` project is created along with Cloud Build triggers, CSRs for application infrastructure code and Google Cloud Storage buckets for state storage.
This step follows the same [conventions](https://github.com/terraform-google-modules/terraform-example-foundation#branching-strategy) as the foundation pipeline deployed in [0-bootstrap](https://github.com/terraform-google-modules/terraform-example-foundation/blob/master/0-bootstrap/README.md).
The Cloud Build SA used by this pipeline can impersonate the project SA by enabling the `enable_cloudbuild_deploy` flag and necessary roles can be granted to this SA via `sa_roles` as shown in this [example](business_unit_1/development/example_base_shared_vpc_project.tf).
This pipeline can be utilized for deploying resources in projects across development/non-production/production with granular permissions.

## Prerequisites

Expand Down Expand Up @@ -94,6 +103,8 @@ Change the `BRANCH_NAME` from `development` to `non-production` or `production`

## Usage

**Note:** You need to set variable `enable_hub_and_spoke` to `true` to be able to used the **Hub-and-Spoke** architecture detailed in the **Networking** section of the [google cloud security foundations guide](https://services.google.com/fh/files/misc/google-cloud-security-foundations-guide.pdf).

### Deploying with Jenkins

1. Clone the repo you created manually in 0-bootstrap.
Expand Down Expand Up @@ -162,7 +173,6 @@ Change the `BRANCH_NAME` from `development` to `non-production` or `production`
git push origin non-production
```
1. Review the apply output in your Master's web UI (you might want to use the option to "Scan Multibranch Pipeline Now" in your Jenkins Master UI).
1. You can now move to the instructions in the step [5-app-infra](../5-app-infra/README.md).
### Deploying with Cloud Build
Expand Down Expand Up @@ -190,8 +200,8 @@ Change the `BRANCH_NAME` from `development` to `non-production` or `production`
```
chmod 755 ./tf-wrapper.sh
```
1. Rename `common.auto.example.tfvars` to `common.auto.tfvars` and update the file with values from your environment and bootstrap.
1. Rename `shared.auto.example.tfvars` to `shared.auto.tfvars` and update the file with values from your environment and bootstrap.
1. Rename `common.auto.example.tfvars` to `common.auto.tfvars` and update the file with values from your environment and bootstrap. See any of the business unit envs folders [README.md](./business_unit_1/development/README.md) files for additional information on the values in the `common.auto.tfvars file`.
1. Rename `shared.auto.example.tfvars` to `shared.auto.tfvars` and update the file with values from your environment and bootstrap. See any of the business unit shared envs folders [README.md](./business_unit_1/shared/README.md) files for additional information on the values in the `shared.auto.example.tfvars`.
1. Rename `development.auto.example.tfvars` to `development.auto.tfvars` and update the file with the `perimeter_name` that starts with `sp_d_shared_restricted`.
1. Rename `non-production.auto.example.tfvars` to `non-production.auto.tfvars` and update the file with the `perimeter_name` that starts with `sp_n_shared_restricted`.
1. Rename `production.auto.example.tfvars` to `production.auto.tfvars` and update the file with the `perimeter_name` that starts with `sp_p_shared_restricted`.
Expand All @@ -202,7 +212,7 @@ Change the `BRANCH_NAME` from `development` to `non-production` or `production`
1. Run `terraform init`.
1. Run `terraform plan` and review output.
1. Run `terraform apply`.
1. Run `terraform output cloudbuild_sa` to get the cloudbuild service account from the apply step.
1. Run `terraform output cloudbuild_sa` to get the cloud build service account from the apply step.
1. If you would like the bucket to be replaced by cloud build at run time, change the bucket name back to `UPDATE_ME`
1. Once you have done the instructions for the `business_unit_1`, you need to repeat same steps for `business_unit_2` folder.
1. Rename `business_unit_1.auto.example.tfvars` to `business_unit_1.auto.tfvars` and update the file with the `app_infra_pipeline_cloudbuild_sa` which is the output of `cloudbuild_sa` from `business_unit_1/shared` steps.
Expand All @@ -223,18 +233,21 @@ Change the `BRANCH_NAME` from `development` to `non-production` or `production`
git push origin production
```
1. Review the apply output in your Cloud Build project. https://console.cloud.google.com/cloud-build/builds?project=YOUR_CLOUD_BUILD_PROJECT_ID
1. After production has been applied, apply development.
1. Merge changes to development.
```
git checkout -b development
git push origin development
```
1. Review the apply output in your Cloud Build project https://console.cloud.google.com/cloud-build/builds?project=YOUR_CLOUD_BUILD_PROJECT_ID
1. After development has been applied, apply non-production.
1. Merge changes to non-production.
```
git checkout -b non-production
git push origin non-production
```
1. Review the apply output in your Cloud Build project. https://console.cloud.google.com/cloud-build/builds?project=YOUR_CLOUD_BUILD_PROJECT_ID
1. You can now move to the instructions in the step [5-app-infra](../5-app-infra/README.md).
### Run Terraform locally
Expand Down Expand Up @@ -275,4 +288,4 @@ To use the `validate` option of the `tf-wrapper.sh` script, the latest version o
1. Run `./tf-wrapper.sh validate development $(pwd)/../policy-library <YOUR_CLOUD_BUILD_PROJECT_ID>` and check for violations.
1. Run `./tf-wrapper.sh apply development`.
If you received any errors or made any changes to the Terraform config or `terraform.tfvars` you must re-run `./tf-wrapper.sh plan <env>` before run `./tf-wrapper.sh apply <env>`.
If you received any errors or made any changes to the Terraform config or `terraform.tfvars` you must re-run `./tf-wrapper.sh plan <env>` before running `./tf-wrapper.sh apply <env>`.
6 changes: 3 additions & 3 deletions 4-projects/business_unit_1/development/README.md
Original file line number Diff line number Diff line change
Expand Up @@ -11,7 +11,7 @@
| budget\_amount | The amount to use as the budget | `number` | `1000` | no |
| enable\_hub\_and\_spoke | Enable Hub-and-Spoke architecture. | `bool` | `false` | no |
| firewall\_enable\_logging | Toggle firewall logging for VPC Firewalls. | `bool` | `true` | no |
| folder\_prefix | Name prefix to use for folders created. | `string` | `"fldr"` | no |
| folder\_prefix | Name prefix to use for folders created. Should be the same in all steps. | `string` | `"fldr"` | no |
| gcs\_bucket\_prefix | Name prefix to be used for GCS Bucket | `string` | `"cmek-encrypted-bucket"` | no |
| key\_name | Name to be used for KMS Key | `string` | `"crypto-key-example"` | no |
| key\_rotation\_period | Rotation period in seconds to be used for KMS Key | `string` | `"7776000s"` | no |
Expand All @@ -20,10 +20,10 @@
| location\_kms | Case-Sensitive Location for KMS Keyring (Should be same region as the GCS Bucket) | `string` | `"us"` | no |
| optional\_fw\_rules\_enabled | Toggle creation of optional firewall rules: IAP SSH, IAP RDP and Internal & Global load balancing health check and load balancing IP ranges. | `bool` | `false` | no |
| org\_id | The organization id for the associated services | `string` | n/a | yes |
| parent\_folder | Optional - if using a folder for testing. | `string` | `""` | no |
| parent\_folder | Optional - for an organization with existing projects or for development/validation. It will place all the example foundation resources under the provided folder instead of the root organization. The value is the numeric folder ID. The folder must already exist. Must be the same value used in previous step. | `string` | `""` | no |
| peering\_module\_depends\_on | List of modules or resources peering module depends on. | `list` | `[]` | no |
| perimeter\_name | Access context manager service perimeter name to attach the restricted svpc project. | `string` | n/a | yes |
| project\_prefix | Name prefix to use for projects created. | `string` | `"prj"` | no |
| project\_prefix | Name prefix to use for projects created. Should be the same in all steps. Max size is 3 characters. | `string` | `"prj"` | no |
| secrets\_prj\_suffix | Name suffix to use for secrets project created. | `string` | `"env-secrets"` | no |
| terraform\_service\_account | Service account email of the account to impersonate to run Terraform | `string` | n/a | yes |
| windows\_activation\_enabled | Enable Windows license activation for Windows workloads. | `bool` | `false` | no |
Expand Down
6 changes: 3 additions & 3 deletions 4-projects/business_unit_1/development/variables.tf
Original file line number Diff line number Diff line change
Expand Up @@ -35,7 +35,7 @@ variable "access_context_manager_policy_id" {
}

variable "parent_folder" {
description = "Optional - if using a folder for testing."
description = "Optional - for an organization with existing projects or for development/validation. It will place all the example foundation resources under the provided folder instead of the root organization. The value is the numeric folder ID. The folder must already exist. Must be the same value used in previous step."
type = string
default = ""
}
Expand Down Expand Up @@ -88,13 +88,13 @@ variable "budget_amount" {
}

variable "project_prefix" {
description = "Name prefix to use for projects created."
description = "Name prefix to use for projects created. Should be the same in all steps. Max size is 3 characters."
type = string
default = "prj"
}

variable "folder_prefix" {
description = "Name prefix to use for folders created."
description = "Name prefix to use for folders created. Should be the same in all steps."
type = string
default = "fldr"
}
Expand Down
6 changes: 3 additions & 3 deletions 4-projects/business_unit_1/non-production/README.md
Original file line number Diff line number Diff line change
Expand Up @@ -11,7 +11,7 @@
| budget\_amount | The amount to use as the budget | `number` | `1000` | no |
| enable\_hub\_and\_spoke | Enable Hub-and-Spoke architecture. | `bool` | `false` | no |
| firewall\_enable\_logging | Toggle firewall logging for VPC Firewalls. | `bool` | `true` | no |
| folder\_prefix | Name prefix to use for folders created. | `string` | `"fldr"` | no |
| folder\_prefix | Name prefix to use for folders created. Should be the same in all steps. | `string` | `"fldr"` | no |
| gcs\_bucket\_prefix | Name prefix to be used for GCS Bucket | `string` | `"cmek-encrypted-bucket"` | no |
| key\_name | Name to be used for KMS Key | `string` | `"crypto-key-example"` | no |
| key\_rotation\_period | Rotation period in seconds to be used for KMS Key | `string` | `"7776000s"` | no |
Expand All @@ -20,10 +20,10 @@
| location\_kms | Case-Sensitive Location for KMS Keyring (Should be same region as the GCS Bucket) | `string` | `"us"` | no |
| optional\_fw\_rules\_enabled | Toggle creation of optional firewall rules: IAP SSH, IAP RDP and Internal & Global load balancing health check and load balancing IP ranges. | `bool` | `false` | no |
| org\_id | The organization id for the associated services | `string` | n/a | yes |
| parent\_folder | Optional - if using a folder for testing. | `string` | `""` | no |
| parent\_folder | Optional - for an organization with existing projects or for development/validation. It will place all the example foundation resources under the provided folder instead of the root organization. The value is the numeric folder ID. The folder must already exist. Must be the same value used in previous step. | `string` | `""` | no |
| peering\_module\_depends\_on | List of modules or resources peering module depends on. | `list` | `[]` | no |
| perimeter\_name | Access context manager service perimeter name to attach the restricted svpc project. | `string` | n/a | yes |
| project\_prefix | Name prefix to use for projects created. | `string` | `"prj"` | no |
| project\_prefix | Name prefix to use for projects created. Should be the same in all steps. Max size is 3 characters. | `string` | `"prj"` | no |
| secrets\_prj\_suffix | Name suffix to use for secrets project created. | `string` | `"env-secrets"` | no |
| terraform\_service\_account | Service account email of the account to impersonate to run Terraform | `string` | n/a | yes |
| windows\_activation\_enabled | Enable Windows license activation for Windows workloads. | `bool` | `false` | no |
Expand Down
6 changes: 3 additions & 3 deletions 4-projects/business_unit_1/non-production/variables.tf
Original file line number Diff line number Diff line change
Expand Up @@ -35,7 +35,7 @@ variable "access_context_manager_policy_id" {
}

variable "parent_folder" {
description = "Optional - if using a folder for testing."
description = "Optional - for an organization with existing projects or for development/validation. It will place all the example foundation resources under the provided folder instead of the root organization. The value is the numeric folder ID. The folder must already exist. Must be the same value used in previous step."
type = string
default = ""
}
Expand Down Expand Up @@ -88,13 +88,13 @@ variable "budget_amount" {
}

variable "project_prefix" {
description = "Name prefix to use for projects created."
description = "Name prefix to use for projects created. Should be the same in all steps. Max size is 3 characters."
type = string
default = "prj"
}

variable "folder_prefix" {
description = "Name prefix to use for folders created."
description = "Name prefix to use for folders created. Should be the same in all steps."
type = string
default = "fldr"
}
Expand Down
6 changes: 3 additions & 3 deletions 4-projects/business_unit_1/production/README.md
Original file line number Diff line number Diff line change
Expand Up @@ -12,7 +12,7 @@
| enable\_hub\_and\_spoke | Enable Hub-and-Spoke architecture. | `bool` | `false` | no |
| env\_code | A short form of the environment field | `string` | `"p"` | no |
| firewall\_enable\_logging | Toggle firewall logging for VPC Firewalls. | `bool` | `true` | no |
| folder\_prefix | Name prefix to use for folders created. | `string` | `"fldr"` | no |
| folder\_prefix | Name prefix to use for folders created. Should be the same in all steps. | `string` | `"fldr"` | no |
| gcs\_bucket\_prefix | Name prefix to be used for GCS Bucket | `string` | `"cmek-encrypted-bucket"` | no |
| key\_name | Name to be used for KMS Key | `string` | `"crypto-key-example"` | no |
| key\_rotation\_period | Rotation period in seconds to be used for KMS Key | `string` | `"7776000s"` | no |
Expand All @@ -21,10 +21,10 @@
| location\_kms | Case-Sensitive Location for KMS Keyring (Should be same region as the GCS Bucket) | `string` | `"us"` | no |
| optional\_fw\_rules\_enabled | Toggle creation of optional firewall rules: IAP SSH, IAP RDP and Internal & Global load balancing health check and load balancing IP ranges. | `bool` | `false` | no |
| org\_id | The organization id for the associated services | `string` | n/a | yes |
| parent\_folder | Optional - if using a folder for testing. | `string` | `""` | no |
| parent\_folder | Optional - for an organization with existing projects or for development/validation. It will place all the example foundation resources under the provided folder instead of the root organization. The value is the numeric folder ID. The folder must already exist. Must be the same value used in previous step. | `string` | `""` | no |
| peering\_module\_depends\_on | List of modules or resources peering module depends on. | `list` | `[]` | no |
| perimeter\_name | Access context manager service perimeter name to attach the restricted svpc project. | `string` | n/a | yes |
| project\_prefix | Name prefix to use for projects created. | `string` | `"prj"` | no |
| project\_prefix | Name prefix to use for projects created. Should be the same in all steps. Max size is 3 characters. | `string` | `"prj"` | no |
| secrets\_prj\_suffix | Name suffix to use for secrets project created. | `string` | `"env-secrets"` | no |
| terraform\_service\_account | Service account email of the account to impersonate to run Terraform | `string` | n/a | yes |
| windows\_activation\_enabled | Enable Windows license activation for Windows workloads. | `bool` | `false` | no |
Expand Down
6 changes: 3 additions & 3 deletions 4-projects/business_unit_1/production/variables.tf
Original file line number Diff line number Diff line change
Expand Up @@ -41,7 +41,7 @@ variable "access_context_manager_policy_id" {
}

variable "parent_folder" {
description = "Optional - if using a folder for testing."
description = "Optional - for an organization with existing projects or for development/validation. It will place all the example foundation resources under the provided folder instead of the root organization. The value is the numeric folder ID. The folder must already exist. Must be the same value used in previous step."
type = string
default = ""
}
Expand Down Expand Up @@ -94,13 +94,13 @@ variable "budget_amount" {
}

variable "project_prefix" {
description = "Name prefix to use for projects created."
description = "Name prefix to use for projects created. Should be the same in all steps. Max size is 3 characters."
type = string
default = "prj"
}

variable "folder_prefix" {
description = "Name prefix to use for folders created."
description = "Name prefix to use for folders created. Should be the same in all steps."
type = string
default = "fldr"
}
Expand Down
Loading

0 comments on commit 9bdf88f

Please sign in to comment.