Skip to content

Latest commit

 

History

History
 
 

0-bootstrap

README.md

0-bootstrap

This repo is part of a multi-part guide that shows how to configure and deploy the example.com reference architecture described in Google Cloud security foundations guide (PDF). The following table lists the parts of the guide.

0-bootstrap (this file) Bootstraps a Google Cloud organization, creating all the required resources and permissions to start using the Cloud Foundation Toolkit (CFT). This step also configures a CI/CD pipeline for foundations code in subsequent stages.
1-org Sets up top level shared folders, monitoring and networking projects, and organization-level logging, and sets baseline security settings through organizational policy.
2-environments Sets up development, non-production, and production environments within the Google Cloud organization that you've created.
3-networks Sets up base and restricted shared VPCs with default DNS, NAT (optional), Private Service networking, VPC service controls, on-premises Dedicated Interconnect, and baseline firewall rules for each environment. Also sets up the global DNS hub.
4-projects Set up a folder structure, projects, and application infrastructure pipeline for applications, which are connected as service projects to the shared VPC created in the previous stage.
5-app-infra Deploy a simple Compute Engine instance in one of the business unit projects using the infra pipeline set up in 4-projects.

For an overview of the architecture and the parts, see the terraform-example-foundation README file.

Purpose

The purpose of this step is to bootstrap a Google Cloud organization, creating all the required resources & permissions to start using the Cloud Foundation Toolkit (CFT). This step also configures a CI/CD pipeline for foundations code in subsequent stages. The CI/CD pipeline can use either Cloud Build and Cloud Source Repos or Jenkins and your own Git repos (which might live on-premises).

Prerequisites

To run the commands described in this document, you need to have the following installed:

  • The Google Cloud SDK version 319.0.0 or later
  • Terraform version 0.13.7.
  • An existing project which the user has access to be used by terraform-validator.

Note: Make sure that you use the same version of Terraform throughout this series. Otherwise, you might experience Terraform state snapshot lock errors.

Also make sure that you've done the following:

  1. Set up a Google Cloud organization.
  2. Set up a Google Cloud billing account.
  3. Created Cloud Identity or Google Workspace (formerly G Suite) groups for organization and billing admins.
  4. Added the user who will use Terraform to the group_org_admins group. They must be in this group, or they won't have roles/resourcemanager.projectCreator access.
  5. For the user who will run the procedures in this document, granted the following roles:
    • The roles/resourcemanager.organizationAdmin role on the Google Cloud organization.
    • The roles/billing.admin role on the billing account.
    • The roles/resourcemanager.folderCreator role.
  6. Ensure that you have requested for sufficient projects quota, as the Terraform scripts will create multiple projects from this point onwards. For more information, please see the FAQ.

If other users need to be able to run these procedures, add them to the group represented by the org_project_creators variable. For more information about the permissions that are required, and the resources that are created, see the organization bootstrap module documentation.

Troubleshooting

Please refer to troubleshooting if you run into issues during this step.

Deploying with Jenkins

If you are using the jenkins_bootstrap sub-module, see README-Jenkins for requirements and instructions on how to run the 0-bootstrap step. Using Jenkins requires a few manual steps, including configuring connectivity with your current Jenkins manager (master) environment.

Deploying with Cloud Build

  1. Go to the 0-bootstrap folder.
  2. Rename terraform.example.tfvars to terraform.tfvars and update the file with values from your environment: 
    mv terraform.example.tfvars terraform.tfvars
  3. Run terraform init.
  4. Run terraform plan and review the output.
  5. To run terraform-validator steps please follow the instructions in the Install Terraform Validator section and install version v0.4.0. You will also need to rename the binary from terraform-validator-<your-platform> to terraform-validator and the terraform-validator binary must be in your PATH.
    1. Run terraform plan -input=false -out bootstrap.tfplan
    2. Run terraform show -json bootstrap.tfplan > bootstrap.json
    3. Run terraform-validator validate bootstrap.json --policy-path="../policy-library" --project <A-VALID-PROJECT-ID> and check for violations (<A-VALID-PROJECT-ID> must be an existing project you have access to, this is necessary because Terraform-validator needs to link resources to a valid Google Cloud Platform project).
  6. Run terraform apply.
  7. Run terraform output terraform_service_account to get the email address of the admin. You need this address in a later procedure.
  8. Run terraform output gcs_bucket_tfstate to get your Google Cloud bucket name from Terraform's state.
  9. Copy the backend: 
    cp backend.tf.example backend.tf
  10. Update backend.tf with the name of your Cloud Storage bucket.
  11. Re-run terraform init. When you're prompted, agree to copy state to Cloud Storage.
  12. (Optional) Run terraform apply to verify that state is configured correctly. You should see no changes from the previous state.

Note 1: The output of terraform-validator will contain lines like

ERROR: logging before flag.Parse: I0413 13:49:49.852283 6380 convert.go:189] unsupported resource: google_billing_account_iam_member

or

ERROR: logging before flag.Parse: I0413 13:49:49.852290 6380 convert.go:183] unknown resource: random_id

These are warnings for resources that are not yet supported or not known by terraform-validator, these are not actual errors.

Note 2: After the deploy, even if you did not receive the project quota error described in the Troubleshooting guide, we recommend that you request 50 additional projects for the service account, terraform_service_account, created in this step.

Running Terraform locally

If you deploy using Cloud Build, the bucket information is replaced in the state backends as a part of the build process when the build is executed by Cloud Build. If you want to execute Terraform locally, you need to add your Cloud Storage bucket to the backend.tf files. You can update all of these files with the following steps:

  1. Go to the terraform-example-foundation directory.
  2. Run the following command: 
    for i in `find -name 'backend.tf'`; do sed -i 's/UPDATE_ME/GCS_BUCKET_NAME/' $i; done
    where GCS_BUCKET_NAME is the name of your bucket from the steps you ran earlier.

Inputs

Name Description Type Default Required
billing_account The ID of the billing account to associate projects with. string n/a yes
bucket_prefix Name prefix to use for state bucket created. string "bkt" no
cloud_source_repos List of Cloud Source Repositories created during bootstrap project build stage for use with Cloud Build. list(string) 
[
  "gcp-org",
  "gcp-environments",
  "gcp-networks",
  "gcp-projects"
]
 no
default_region Default region to create resources where applicable. string "asia-southeast1" no
folder_prefix Name prefix to use for folders created. Should be the same in all steps. string "fldr" no
group_billing_admins Google Group for GCP Billing Administrators string n/a yes
group_org_admins Google Group for GCP Organization Administrators string n/a yes
org_id GCP Organization ID string n/a yes
org_policy_admin_role Additional Org Policy Admin role for admin group. You can use this for testing purposes. bool false no
org_project_creators Additional list of members to have project creator role across the organization. Prefix of group: user: or serviceAccount: is required. list(string) [] no
parent_folder Optional - for an organization with existing projects or for development/validation. It will place all the example foundation resources under the provided folder instead of the root organization. The value is the numeric folder ID. The folder must already exist. string "" no
project_prefix Name prefix to use for projects created. Should be the same in all steps. Max size is 3 characters. string "prj" no

Outputs

Name Description
cloudbuild_project_id Project where CloudBuild configuration and terraform container image will reside.
csr_repos List of Cloud Source Repos created by the module, linked to Cloud Build triggers.
gcs_bucket_cloudbuild_artifacts Bucket used to store Cloud/Build artifacts in CloudBuild project.
gcs_bucket_tfstate Bucket used for storing terraform state for foundations pipelines in seed project.
kms_crypto_key KMS key created by the module.
kms_keyring KMS Keyring created by the module.
seed_project_id Project where service accounts and core APIs will be enabled.
terraform_sa_name Fully qualified name for privileged service account for Terraform.
terraform_service_account Email for privileged service account for Terraform.
terraform_validator_policies_repo Cloud Source Repository created for terraform-validator policies.