[PFMENG-1371] Feat/vault ent secret sync

[hazmei]

Special thanks to @shisheng10 and @uchinda-sph for the sharing session of their POC, issues faced and use case of secret sync.

This PR adds the following resources:

1. AWS IAM User
2. AWS IAM Group
3. AWS IAM Policy
4. AWS Access Keys
5. Time rotating resource for rotating access keys (90 days)
6. Vault Enterprise Secret Sync Destination
7. Vault Enterprise Secret Sync Association

Previously tested on web2-platform-infra webcore dev environment.

This setup provide a way to unassociate some vault secret from the secret sync as well as removing the secret sync cleanly (requires 2 step approach for removing the secret sync destination and association).

Current code only supports AWS destination for now. Will add more destinations in the future.

Note

The secret name in aws secrets manager is not configurable. It follows the following format: vault/<vault accessor id>/<vault secret name>.

There's a need to have different vault_generic_endpoint resource for associating and disassociating the vault secret as the endpoint for deleting association doesn't support the DELETE method. See the doc here.

terraform-vault-secret-sync/main.tf

Lines 145 to 178 in 5dddf82

	resource "vault_generic_endpoint" "create_association_sync" {
	for_each = { for secret in local.associate_secrets : "${secret.app_name}-${secret.secret_name}" => secret }
	path = "${local.sync_base_path}/aws-sm/${local.destination_name}/associations/set"
	data_json = jsonencode({
	mount = each.value.mount
	secret_name = each.value.secret_name
	})
	disable_delete = true
	disable_read = true
	ignore_absent_fields = true
	depends_on = [
	time_sleep.wait_5_seconds,
	]
	}
	# Remove Some Vault -> AWS SM association
	resource "vault_generic_endpoint" "remove_some_association_sync" {
	for_each = { for secret in local.unassociate_secrets : "${secret.app_name}-${secret.secret_name}" => secret }
	path = "${local.sync_base_path}/aws-sm/${local.destination_name}/associations/remove"
	data_json = jsonencode({
	mount = each.value.mount
	secret_name = each.value.secret_name
	})
	disable_delete = true
	disable_read = true
	ignore_absent_fields = true
	}

Creating secret sync destination and association

module "vault_secret_sync"
  source  = "app.terraform.io/sph/terraform-vault-secret-sync" # this is just an example
  version = "0.1.0-alpha" # this is just an example

  name = "webcore-dev"

  associate_secrets = {
    web2 = {
      mount       = "kvv2"
      secret_name = ["secret1","secret2"]
    }
  }

Removing secret association

module "vault_secret_sync"
  source  = "app.terraform.io/sph/terraform-vault-secret-sync" # this is just an example
  version = "0.1.0-alpha" # this is just an example

  name = "webcore-dev"

  associate_secrets = {
    web2 = {
      mount       = "kvv2"
      secret_name = ["secret1"]
    }
  }

  unassociate_secrets = {
    web2 = {
      mount       = "kvv2"
      secret_name = ["secret2"]
    }
  }

Removing the secret sync destination and secret association

This requires a 2 step flow to remove the destination and association. Secret associations must be removed before the destination can be removed.

Step 1

module "vault_secret_sync"
  source  = "app.terraform.io/sph/terraform-vault-secret-sync" # this is just an example
  version = "0.1.0-alpha" # this is just an example

  name = "webcore-dev"

  associate_secrets = {
    web2 = {
      mount       = "kvv2"
      secret_name = ["secret1","secret2"]
    }
  }

  delete_all_secrets_associations = true

Step 2

module "vault_secret_sync"
  source  = "app.terraform.io/sph/terraform-vault-secret-sync" # this is just an example
  version = "0.1.0-alpha" # this is just an example

  name = "webcore-dev"

  associate_secrets = {
    web2 = {
      mount       = "kvv2"
      secret_name = ["secret1","secret2"]
    }
  }

  delete_all_secrets_associations = true
  delete_sync_destination         = true

[github-advanced-security]

This pull request sets up GitHub code scanning for this repository. Once the scans have completed and the checks have passed, the analysis results for this pull request branch will appear on this overview. Once you merge this pull request, the 'Security' tab will show more code scanning analysis results (for example, for the default branch). Depending on your configuration and choice of analysis tool, future pull requests will be annotated with code scanning analysis results. For more information about GitHub code scanning, check out the documentation.

[shisheng10]

LGTM

[niroz89]

@hazmei add the LICENSE as well, let's publish to public registry.