👁️ Observability: Add Athena Data Source #8142

Gary-H9 · 2024-10-09T13:31:43Z

Raised in relation to this piece of work.

Raised in conjunction with this PR in the terraform-aws-observability-platform-tenant repository.

2.5k lines of Terraform Static Analysis is borderline unreadable but from what I can see it's not related to the work here.

github-actions · 2024-10-17T15:51:24Z

`Trivy Scan` Success

Show Output

```hcl

Trivy will check the following folders:
terraform/environments/observability-platform
terraform/environments/observability-platform/modules/grafana/athena-source
terraform/environments/observability-platform/modules/grafana/team
terraform/environments/observability-platform/modules/observability-platform/tenant-configuration

Running Trivy in terraform/environments/observability-platform
2024-10-17T15:51:02Z INFO [vulndb] Need to update DB
2024-10-17T15:51:02Z INFO [vulndb] Downloading vulnerability DB...
2024-10-17T15:51:02Z INFO [vulndb] Downloading artifact... repo="ghcr.io/aquasecurity/trivy-db:2"
2024-10-17T15:51:04Z INFO [vulndb] Artifact successfully downloaded repo="ghcr.io/aquasecurity/trivy-db:2"
2024-10-17T15:51:04Z INFO [vuln] Vulnerability scanning is enabled
2024-10-17T15:51:04Z INFO [misconfig] Misconfiguration scanning is enabled
2024-10-17T15:51:04Z INFO [misconfig] Need to update the built-in checks
2024-10-17T15:51:04Z INFO [misconfig] Downloading the built-in checks...
2024-10-17T15:51:04Z ERROR [misconfig] Falling back to embedded checks err="failed to download built-in policies: download error: OCI repository error: 1 error occurred:\n\t* GET https://ghcr.io/v2/aquasecurity/trivy-checks/manifests/1: TOOMANYREQUESTS: retry-after: 1.102685ms, allowed: 44000/minute\n\n"
2024-10-17T15:51:04Z INFO [secret] Secret scanning is enabled
2024-10-17T15:51:04Z INFO [secret] If your scanning is slow, please try '--scanners vuln' to disable secret scanning
2024-10-17T15:51:04Z INFO [secret] Please see also https://aquasecurity.github.io/trivy/v0.56/docs/scanner/secret#recommendation for faster secret detection
2024-10-17T15:51:05Z INFO [terraform scanner] Scanning root module file_path="."
2024-10-17T15:51:05Z WARN [terraform parser] Variable values was not found in the environment or variable files. Evaluating may not work correctly. module="root" variables="networking"
2024-10-17T15:51:05Z ERROR [terraform evaluator] Failed to expand block. Invalid "for-each" argument. Must be known and iterable. block="data.aws_identitystore_group.all_identity_centre_teams" value="cty.NilVal"
2024-10-17T15:51:05Z ERROR [terraform evaluator] Failed to expand block. Invalid "for-each" argument. Must be known and iterable. block="module.contact_point_pagerduty" value="cty.NilVal"
2024-10-17T15:51:05Z ERROR [terraform evaluator] Failed to expand block. Invalid "for-each" argument. Must be known and iterable. block="module.contact_point_slack" value="cty.NilVal"
2024-10-17T15:51:05Z ERROR [terraform evaluator] Failed to expand block. Invalid "for-each" argument. Must be known and iterable. block="module.tenant_configuration" value="cty.NilVal"
2024-10-17T15:51:05Z ERROR [terraform evaluator] Failed to expand block. Invalid "for-each" argument. Must be known and iterable. block="dynamic.policy" value="cty.NilVal"
2024-10-17T15:51:05Z ERROR [terraform evaluator] Failed to expand block. Invalid "for-each" argument. Must be known and iterable. block="dynamic.policy" value="cty.NilVal"
2024-10-17T15:51:05Z ERROR [terraform evaluator] Failed to expand block. Invalid "for-each" argument. Must be known and iterable. block="dynamic.policy" value="cty.NilVal"
2024-10-17T15:51:05Z ERROR [terraform evaluator] Failed to expand block. Invalid "for-each" argument. Must be known and iterable. block="dynamic.policy" value="cty.NilVal"
2024-10-17T15:51:07Z ERROR [terraform evaluator] Failed to expand block. Invalid "for-each" argument. Must be known and iterable. block="module.grafana_api_key_rotator.dynamic.logging_config" value="cty.NilVal"
2024-10-17T15:51:07Z ERROR [terraform evaluator] Failed to expand block. Invalid "for-each" argument. Must be known and iterable. block="module.grafana_api_key_rotator.dynamic.logging_config" value="cty.NilVal"
2024-10-17T15:51:07Z ERROR [terraform evaluator] Failed to expand block. Invalid "for-each" argument. Must be known and iterable. block="module.grafana_api_key_rotator.dynamic.logging_config" value="cty.NilVal"
2024-10-17T15:51:07Z ERROR [terraform evaluator] Failed to expand block. Invalid "for-each" argument. Must be known and iterable. block="module.grafana_api_key_rotator.dynamic.logging_config" value="cty.NilVal"
2024-10-17T15:51:08Z INFO [terraform executor] Ignore finding rule="aws-lambda-enable-tracing" range="git::https:/github.com/terraform-aws-modules/terraform-aws-lambda?ref=f48be17ec03a53b85b7da1f2ad8787792f2425ee/main.tf:24-166"
2024-10-17T15:51:08Z INFO [terraform executor] Ignore finding rule="aws-ssm-secret-use-customer-key" range="secrets.tf:2-7"
2024-10-17T15:51:08Z INFO [terraform executor] Ignore finding rule="aws-ssm-secret-use-customer-key" range="secrets.tf:10-15"
2024-10-17T15:51:08Z INFO [terraform executor] Ignore finding rule="aws-ssm-secret-use-customer-key" range="secrets.tf:18-23"
2024-10-17T15:51:08Z INFO [terraform executor] Ignore finding rule="aws-ssm-secret-use-customer-key" range="secrets.tf:26-31"
2024-10-17T15:51:08Z INFO Number of language-specific files num=0
2024-10-17T15:51:08Z INFO Detected config files num=3
trivy_exitcode=0

Running Trivy in terraform/environments/observability-platform/modules/grafana/athena-source
2024-10-17T15:51:08Z INFO [vuln] Vulnerability scanning is enabled
2024-10-17T15:51:08Z INFO [misconfig] Misconfiguration scanning is enabled
2024-10-17T15:51:08Z INFO [misconfig] Need to update the built-in checks
2024-10-17T15:51:08Z INFO [misconfig] Downloading the built-in checks...
2024-10-17T15:51:08Z ERROR [misconfig] Falling back to embedded checks err="failed to download built-in policies: download error: OCI repository error: 1 error occurred:\n\t* GET https://ghcr.io/v2/aquasecurity/trivy-checks/manifests/1: TOOMANYREQUESTS: retry-after: 349.64µs, allowed: 44000/minute\n\n"
2024-10-17T15:51:08Z INFO [secret] Secret scanning is enabled
2024-10-17T15:51:08Z INFO [secret] If your scanning is slow, please try '--scanners vuln' to disable secret scanning
2024-10-17T15:51:08Z INFO [secret] Please see also https://aquasecurity.github.io/trivy/v0.56/docs/scanner/secret#recommendation for faster secret detection
2024-10-17T15:51:08Z INFO [terraform scanner] Scanning root module file_path="."
2024-10-17T15:51:08Z WARN [terraform parser] Variable values was not found in the environment or variable files. Evaluating may not work correctly. module="root" variables="account_id, athena_config, name"
2024-10-17T15:51:08Z ERROR [terraform evaluator] Failed to expand block. Invalid "for-each" argument. Must be known and iterable. block="grafana_data_source.this" value="cty.NilVal"
2024-10-17T15:51:09Z INFO Number of language-specific files num=0
2024-10-17T15:51:09Z INFO Detected config files num=1
trivy_exitcode=0

Running Trivy in terraform/environments/observability-platform/modules/grafana/team
2024-10-17T15:51:09Z INFO [vuln] Vulnerability scanning is enabled
2024-10-17T15:51:09Z INFO [misconfig] Misconfiguration scanning is enabled
2024-10-17T15:51:09Z INFO [misconfig] Need to update the built-in checks
2024-10-17T15:51:09Z INFO [misconfig] Downloading the built-in checks...
156.02 KiB / 156.02 KiB [------------------------------------------------------] 100.00% ? p/s 100ms2024-10-17T15:51:09Z INFO [secret] Secret scanning is enabled
2024-10-17T15:51:09Z INFO [secret] If your scanning is slow, please try '--scanners vuln' to disable secret scanning
2024-10-17T15:51:09Z INFO [secret] Please see also https://aquasecurity.github.io/trivy/v0.56/docs/scanner/secret#recommendation for faster secret detection
2024-10-17T15:51:10Z INFO [terraform scanner] Scanning root module file_path="."
2024-10-17T15:51:10Z WARN [terraform parser] Variable values was not found in the environment or variable files. Evaluating may not work correctly. module="root" variables="aws_accounts, identity_centre_team, name"
2024-10-17T15:51:10Z ERROR [terraform evaluator] Failed to expand block. Invalid "for-each" argument. Must be known and iterable. block="data.grafana_data_source.amazon_prometheus" value="cty.NilVal"
2024-10-17T15:51:10Z ERROR [terraform evaluator] Failed to expand block. Invalid "for-each" argument. Must be known and iterable. block="data.grafana_data_source.athena" value="cty.NilVal"
2024-10-17T15:51:10Z ERROR [terraform evaluator] Failed to expand block. Invalid "for-each" argument. Must be known and iterable. block="data.grafana_data_source.cloudwatch" value="cty.NilVal"
2024-10-17T15:51:10Z ERROR [terraform evaluator] Failed to expand block. Invalid "for-each" argument. Must be known and iterable. block="data.grafana_data_source.xray" value="cty.NilVal"
2024-10-17T15:51:10Z ERROR [terraform evaluator] Failed to expand block. Invalid "for-each" argument. Must be known and iterable. block="grafana_data_source_permission.amazon_prometheus" value="cty.NilVal"
2024-10-17T15:51:10Z ERROR [terraform evaluator] Failed to expand block. Invalid "for-each" argument. Must be known and iterable. block="grafana_data_source_permission.athena" value="cty.NilVal"
2024-10-17T15:51:10Z ERROR [terraform evaluator] Failed to expand block. Invalid "for-each" argument. Must be known and iterable. block="grafana_data_source_permission.cloudwatch" value="cty.NilVal"
2024-10-17T15:51:10Z ERROR [terraform evaluator] Failed to expand block. Invalid "for-each" argument. Must be known and iterable. block="grafana_data_source_permission.xray" value="cty.NilVal"
2024-10-17T15:51:10Z INFO Number of language-specific files num=0
2024-10-17T15:51:10Z INFO Detected config files num=1
trivy_exitcode=0

Running Trivy in terraform/environments/observability-platform/modules/observability-platform/tenant-configuration
2024-10-17T15:51:10Z INFO [vuln] Vulnerability scanning is enabled
2024-10-17T15:51:10Z INFO [misconfig] Misconfiguration scanning is enabled
2024-10-17T15:51:10Z INFO [secret] Secret scanning is enabled
2024-10-17T15:51:10Z INFO [secret] If your scanning is slow, please try '--scanners vuln' to disable secret scanning
2024-10-17T15:51:10Z INFO [secret] Please see also https://aquasecurity.github.io/trivy/v0.56/docs/scanner/secret#recommendation for faster secret detection
2024-10-17T15:51:11Z INFO [terraform scanner] Scanning root module file_path="."
2024-10-17T15:51:11Z WARN [terraform parser] Variable values was not found in the environment or variable files. Evaluating may not work correctly. module="root" variables="environment_management, identity_centre_team, name"
2024-10-17T15:51:11Z INFO Number of language-specific files num=0
2024-10-17T15:51:11Z INFO Detected config files num=1
trivy_exitcode=0

</details> #### `Checkov Scan` Success
<details><summary>Show Output</summary>

```hcl

*****************************

Checkov will check the following folders:
terraform/environments/observability-platform
terraform/environments/observability-platform/modules/grafana/athena-source
terraform/environments/observability-platform/modules/grafana/team
terraform/environments/observability-platform/modules/observability-platform/tenant-configuration

*****************************

Running Checkov in terraform/environments/observability-platform
Excluding the following checks: CKV_GIT_1,CKV_AWS_126,CKV2_AWS_38,CKV2_AWS_39
2024-10-17 15:51:13,708 [MainThread  ] [WARNI]  Failed to download module ministryofjustice/observability-platform-tenant/aws:1.2.0 (for external modules, the --download-external-modules flag is required)
2024-10-17 15:51:13,709 [MainThread  ] [WARNI]  Failed to download module terraform-aws-modules/lambda/aws:7.7.1 (for external modules, the --download-external-modules flag is required)
2024-10-17 15:51:13,709 [MainThread  ] [WARNI]  Failed to download module terraform-aws-modules/managed-service-prometheus/aws:2.2.3 (for external modules, the --download-external-modules flag is required)
2024-10-17 15:51:13,709 [MainThread  ] [WARNI]  Failed to download module terraform-aws-modules/managed-service-grafana/aws:2.1.1 (for external modules, the --download-external-modules flag is required)
2024-10-17 15:51:13,709 [MainThread  ] [WARNI]  Failed to download module terraform-aws-modules/iam/aws//modules/iam-policy:5.44.0 (for external modules, the --download-external-modules flag is required)
2024-10-17 15:51:13,709 [MainThread  ] [WARNI]  Failed to download module terraform-aws-modules/iam/aws//modules/iam-assumable-role:5.44.0 (for external modules, the --download-external-modules flag is required)
terraform scan results:

Passed checks: 30, Failed checks: 0, Skipped checks: 22


checkov_exitcode=0

*****************************

Running Checkov in terraform/environments/observability-platform/modules/grafana/athena-source
Excluding the following checks: CKV_GIT_1,CKV_AWS_126,CKV2_AWS_38,CKV2_AWS_39

checkov_exitcode=0

*****************************

Running Checkov in terraform/environments/observability-platform/modules/grafana/team
Excluding the following checks: CKV_GIT_1,CKV_AWS_126,CKV2_AWS_38,CKV2_AWS_39

checkov_exitcode=0

*****************************

Running Checkov in terraform/environments/observability-platform/modules/observability-platform/tenant-configuration
Excluding the following checks: CKV_GIT_1,CKV_AWS_126,CKV2_AWS_38,CKV2_AWS_39
terraform scan results:

Passed checks: 11, Failed checks: 0, Skipped checks: 4


checkov_exitcode=0

`CTFLint Scan` Success

Show Output

*****************************

Setting default tflint config...
Running tflint --init...
Installing "terraform" plugin...
Installed "terraform" (source: github.com/terraform-linters/tflint-ruleset-terraform, version: 0.9.1)
tflint will check the following folders:
terraform/environments/observability-platform
terraform/environments/observability-platform/modules/grafana/athena-source
terraform/environments/observability-platform/modules/grafana/team
terraform/environments/observability-platform/modules/observability-platform/tenant-configuration

*****************************

Running tflint in terraform/environments/observability-platform
Excluding the following checks: terraform_unused_declarations
tflint_exitcode=0

*****************************

Running tflint in terraform/environments/observability-platform/modules/grafana/athena-source
Excluding the following checks: terraform_unused_declarations
tflint_exitcode=0

*****************************

Running tflint in terraform/environments/observability-platform/modules/grafana/team
Excluding the following checks: terraform_unused_declarations
tflint_exitcode=0

*****************************

Running tflint in terraform/environments/observability-platform/modules/observability-platform/tenant-configuration
Excluding the following checks: terraform_unused_declarations
tflint_exitcode=0

`Trivy Scan` Success

Show Output

*****************************

Trivy will check the following folders:
terraform/environments/observability-platform
terraform/environments/observability-platform/modules/grafana/athena-source
terraform/environments/observability-platform/modules/grafana/team
terraform/environments/observability-platform/modules/observability-platform/tenant-configuration

*****************************

Running Trivy in terraform/environments/observability-platform
2024-10-17T15:51:02Z	INFO	[vulndb] Need to update DB
2024-10-17T15:51:02Z	INFO	[vulndb] Downloading vulnerability DB...
2024-10-17T15:51:02Z	INFO	[vulndb] Downloading artifact...	repo="ghcr.io/aquasecurity/trivy-db:2"
2024-10-17T15:51:04Z	INFO	[vulndb] Artifact successfully downloaded	repo="ghcr.io/aquasecurity/trivy-db:2"
2024-10-17T15:51:04Z	INFO	[vuln] Vulnerability scanning is enabled
2024-10-17T15:51:04Z	INFO	[misconfig] Misconfiguration scanning is enabled
2024-10-17T15:51:04Z	INFO	[misconfig] Need to update the built-in checks
2024-10-17T15:51:04Z	INFO	[misconfig] Downloading the built-in checks...
2024-10-17T15:51:04Z	ERROR	[misconfig] Falling back to embedded checks	err="failed to download built-in policies: download error: OCI repository error: 1 error occurred:\n\t* GET https://ghcr.io/v2/aquasecurity/trivy-checks/manifests/1: TOOMANYREQUESTS: retry-after: 1.102685ms, allowed: 44000/minute\n\n"
2024-10-17T15:51:04Z	INFO	[secret] Secret scanning is enabled
2024-10-17T15:51:04Z	INFO	[secret] If your scanning is slow, please try '--scanners vuln' to disable secret scanning
2024-10-17T15:51:04Z	INFO	[secret] Please see also https://aquasecurity.github.io/trivy/v0.56/docs/scanner/secret#recommendation for faster secret detection
2024-10-17T15:51:05Z	INFO	[terraform scanner] Scanning root module	file_path="."
2024-10-17T15:51:05Z	WARN	[terraform parser] Variable values was not found in the environment or variable files. Evaluating may not work correctly.	module="root" variables="networking"
2024-10-17T15:51:05Z	ERROR	[terraform evaluator] Failed to expand block. Invalid "for-each" argument. Must be known and iterable.	block="data.aws_identitystore_group.all_identity_centre_teams" value="cty.NilVal"
2024-10-17T15:51:05Z	ERROR	[terraform evaluator] Failed to expand block. Invalid "for-each" argument. Must be known and iterable.	block="module.contact_point_pagerduty" value="cty.NilVal"
2024-10-17T15:51:05Z	ERROR	[terraform evaluator] Failed to expand block. Invalid "for-each" argument. Must be known and iterable.	block="module.contact_point_slack" value="cty.NilVal"
2024-10-17T15:51:05Z	ERROR	[terraform evaluator] Failed to expand block. Invalid "for-each" argument. Must be known and iterable.	block="module.tenant_configuration" value="cty.NilVal"
2024-10-17T15:51:05Z	ERROR	[terraform evaluator] Failed to expand block. Invalid "for-each" argument. Must be known and iterable.	block="dynamic.policy" value="cty.NilVal"
2024-10-17T15:51:05Z	ERROR	[terraform evaluator] Failed to expand block. Invalid "for-each" argument. Must be known and iterable.	block="dynamic.policy" value="cty.NilVal"
2024-10-17T15:51:05Z	ERROR	[terraform evaluator] Failed to expand block. Invalid "for-each" argument. Must be known and iterable.	block="dynamic.policy" value="cty.NilVal"
2024-10-17T15:51:05Z	ERROR	[terraform evaluator] Failed to expand block. Invalid "for-each" argument. Must be known and iterable.	block="dynamic.policy" value="cty.NilVal"
2024-10-17T15:51:07Z	ERROR	[terraform evaluator] Failed to expand block. Invalid "for-each" argument. Must be known and iterable.	block="module.grafana_api_key_rotator.dynamic.logging_config" value="cty.NilVal"
2024-10-17T15:51:07Z	ERROR	[terraform evaluator] Failed to expand block. Invalid "for-each" argument. Must be known and iterable.	block="module.grafana_api_key_rotator.dynamic.logging_config" value="cty.NilVal"
2024-10-17T15:51:07Z	ERROR	[terraform evaluator] Failed to expand block. Invalid "for-each" argument. Must be known and iterable.	block="module.grafana_api_key_rotator.dynamic.logging_config" value="cty.NilVal"
2024-10-17T15:51:07Z	ERROR	[terraform evaluator] Failed to expand block. Invalid "for-each" argument. Must be known and iterable.	block="module.grafana_api_key_rotator.dynamic.logging_config" value="cty.NilVal"
2024-10-17T15:51:08Z	INFO	[terraform executor] Ignore finding	rule="aws-lambda-enable-tracing" range="git::https:/github.com/terraform-aws-modules/terraform-aws-lambda?ref=f48be17ec03a53b85b7da1f2ad8787792f2425ee/main.tf:24-166"
2024-10-17T15:51:08Z	INFO	[terraform executor] Ignore finding	rule="aws-ssm-secret-use-customer-key" range="secrets.tf:2-7"
2024-10-17T15:51:08Z	INFO	[terraform executor] Ignore finding	rule="aws-ssm-secret-use-customer-key" range="secrets.tf:10-15"
2024-10-17T15:51:08Z	INFO	[terraform executor] Ignore finding	rule="aws-ssm-secret-use-customer-key" range="secrets.tf:18-23"
2024-10-17T15:51:08Z	INFO	[terraform executor] Ignore finding	rule="aws-ssm-secret-use-customer-key" range="secrets.tf:26-31"
2024-10-17T15:51:08Z	INFO	Number of language-specific files	num=0
2024-10-17T15:51:08Z	INFO	Detected config files	num=3
trivy_exitcode=0

*****************************

Running Trivy in terraform/environments/observability-platform/modules/grafana/athena-source
2024-10-17T15:51:08Z	INFO	[vuln] Vulnerability scanning is enabled
2024-10-17T15:51:08Z	INFO	[misconfig] Misconfiguration scanning is enabled
2024-10-17T15:51:08Z	INFO	[misconfig] Need to update the built-in checks
2024-10-17T15:51:08Z	INFO	[misconfig] Downloading the built-in checks...
2024-10-17T15:51:08Z	ERROR	[misconfig] Falling back to embedded checks	err="failed to download built-in policies: download error: OCI repository error: 1 error occurred:\n\t* GET https://ghcr.io/v2/aquasecurity/trivy-checks/manifests/1: TOOMANYREQUESTS: retry-after: 349.64µs, allowed: 44000/minute\n\n"
2024-10-17T15:51:08Z	INFO	[secret] Secret scanning is enabled
2024-10-17T15:51:08Z	INFO	[secret] If your scanning is slow, please try '--scanners vuln' to disable secret scanning
2024-10-17T15:51:08Z	INFO	[secret] Please see also https://aquasecurity.github.io/trivy/v0.56/docs/scanner/secret#recommendation for faster secret detection
2024-10-17T15:51:08Z	INFO	[terraform scanner] Scanning root module	file_path="."
2024-10-17T15:51:08Z	WARN	[terraform parser] Variable values was not found in the environment or variable files. Evaluating may not work correctly.	module="root" variables="account_id, athena_config, name"
2024-10-17T15:51:08Z	ERROR	[terraform evaluator] Failed to expand block. Invalid "for-each" argument. Must be known and iterable.	block="grafana_data_source.this" value="cty.NilVal"
2024-10-17T15:51:09Z	INFO	Number of language-specific files	num=0
2024-10-17T15:51:09Z	INFO	Detected config files	num=1
trivy_exitcode=0

*****************************

Running Trivy in terraform/environments/observability-platform/modules/grafana/team
2024-10-17T15:51:09Z	INFO	[vuln] Vulnerability scanning is enabled
2024-10-17T15:51:09Z	INFO	[misconfig] Misconfiguration scanning is enabled
2024-10-17T15:51:09Z	INFO	[misconfig] Need to update the built-in checks
2024-10-17T15:51:09Z	INFO	[misconfig] Downloading the built-in checks...
156.02 KiB / 156.02 KiB [------------------------------------------------------] 100.00% ? p/s 100ms2024-10-17T15:51:09Z	INFO	[secret] Secret scanning is enabled
2024-10-17T15:51:09Z	INFO	[secret] If your scanning is slow, please try '--scanners vuln' to disable secret scanning
2024-10-17T15:51:09Z	INFO	[secret] Please see also https://aquasecurity.github.io/trivy/v0.56/docs/scanner/secret#recommendation for faster secret detection
2024-10-17T15:51:10Z	INFO	[terraform scanner] Scanning root module	file_path="."
2024-10-17T15:51:10Z	WARN	[terraform parser] Variable values was not found in the environment or variable files. Evaluating may not work correctly.	module="root" variables="aws_accounts, identity_centre_team, name"
2024-10-17T15:51:10Z	ERROR	[terraform evaluator] Failed to expand block. Invalid "for-each" argument. Must be known and iterable.	block="data.grafana_data_source.amazon_prometheus" value="cty.NilVal"
2024-10-17T15:51:10Z	ERROR	[terraform evaluator] Failed to expand block. Invalid "for-each" argument. Must be known and iterable.	block="data.grafana_data_source.athena" value="cty.NilVal"
2024-10-17T15:51:10Z	ERROR	[terraform evaluator] Failed to expand block. Invalid "for-each" argument. Must be known and iterable.	block="data.grafana_data_source.cloudwatch" value="cty.NilVal"
2024-10-17T15:51:10Z	ERROR	[terraform evaluator] Failed to expand block. Invalid "for-each" argument. Must be known and iterable.	block="data.grafana_data_source.xray" value="cty.NilVal"
2024-10-17T15:51:10Z	ERROR	[terraform evaluator] Failed to expand block. Invalid "for-each" argument. Must be known and iterable.	block="grafana_data_source_permission.amazon_prometheus" value="cty.NilVal"
2024-10-17T15:51:10Z	ERROR	[terraform evaluator] Failed to expand block. Invalid "for-each" argument. Must be known and iterable.	block="grafana_data_source_permission.athena" value="cty.NilVal"
2024-10-17T15:51:10Z	ERROR	[terraform evaluator] Failed to expand block. Invalid "for-each" argument. Must be known and iterable.	block="grafana_data_source_permission.cloudwatch" value="cty.NilVal"
2024-10-17T15:51:10Z	ERROR	[terraform evaluator] Failed to expand block. Invalid "for-each" argument. Must be known and iterable.	block="grafana_data_source_permission.xray" value="cty.NilVal"
2024-10-17T15:51:10Z	INFO	Number of language-specific files	num=0
2024-10-17T15:51:10Z	INFO	Detected config files	num=1
trivy_exitcode=0

*****************************

Running Trivy in terraform/environments/observability-platform/modules/observability-platform/tenant-configuration
2024-10-17T15:51:10Z	INFO	[vuln] Vulnerability scanning is enabled
2024-10-17T15:51:10Z	INFO	[misconfig] Misconfiguration scanning is enabled
2024-10-17T15:51:10Z	INFO	[secret] Secret scanning is enabled
2024-10-17T15:51:10Z	INFO	[secret] If your scanning is slow, please try '--scanners vuln' to disable secret scanning
2024-10-17T15:51:10Z	INFO	[secret] Please see also https://aquasecurity.github.io/trivy/v0.56/docs/scanner/secret#recommendation for faster secret detection
2024-10-17T15:51:11Z	INFO	[terraform scanner] Scanning root module	file_path="."
2024-10-17T15:51:11Z	WARN	[terraform parser] Variable values was not found in the environment or variable files. Evaluating may not work correctly.	module="root" variables="environment_management, identity_centre_team, name"
2024-10-17T15:51:11Z	INFO	Number of language-specific files	num=0
2024-10-17T15:51:11Z	INFO	Detected config files	num=1
trivy_exitcode=0

github-actions · 2024-10-21T14:32:10Z

`Trivy Scan` Failed

Show Output

```hcl

Trivy will check the following folders:
terraform/environments/observability-platform
terraform/environments/observability-platform/modules/grafana/athena-source
terraform/environments/observability-platform/modules/grafana/team
terraform/environments/observability-platform/modules/observability-platform/tenant-configuration

Running Trivy in terraform/environments/observability-platform
2024-10-21T14:31:56Z INFO [vulndb] Need to update DB
2024-10-21T14:31:56Z INFO [vulndb] Downloading vulnerability DB...
2024-10-21T14:31:56Z INFO [vulndb] Downloading artifact... repo="ghcr.io/aquasecurity/trivy-db:2"
2024-10-21T14:31:56Z ERROR [vulndb] Failed to download artifact repo="ghcr.io/aquasecurity/trivy-db:2" err="oci download error: failed to fetch the layer: GET https://ghcr.io/v2/aquasecurity/trivy-db/blobs/sha256:07994604d8e647483cf5193669d01210524ae223c43ca96364d63f59bd64d9bf: TOOMANYREQUESTS: retry-after: 4.291µs, allowed: 44000/minute"
2024-10-21T14:31:56Z FATAL Fatal error init error: DB error: failed to download vulnerability DB: OCI artifact error: failed to download vulnerability DB: failed to download artifact from any source
trivy_exitcode=1

Running Trivy in terraform/environments/observability-platform/modules/grafana/athena-source
2024-10-21T14:31:56Z INFO [vulndb] Need to update DB
2024-10-21T14:31:56Z INFO [vulndb] Downloading vulnerability DB...
2024-10-21T14:31:56Z INFO [vulndb] Downloading artifact... repo="ghcr.io/aquasecurity/trivy-db:2"
2024-10-21T14:31:56Z ERROR [vulndb] Failed to download artifact repo="ghcr.io/aquasecurity/trivy-db:2" err="OCI repository error: 1 error occurred:\n\t* GET https://ghcr.io/v2/aquasecurity/trivy-db/manifests/2: TOOMANYREQUESTS: retry-after: 621.616µs, allowed: 44000/minute\n\n"
2024-10-21T14:31:56Z FATAL Fatal error init error: DB error: failed to download vulnerability DB: OCI artifact error: failed to download vulnerability DB: failed to download artifact from any source
trivy_exitcode=2

Running Trivy in terraform/environments/observability-platform/modules/grafana/team
2024-10-21T14:31:56Z INFO [vulndb] Need to update DB
2024-10-21T14:31:56Z INFO [vulndb] Downloading vulnerability DB...
2024-10-21T14:31:56Z INFO [vulndb] Downloading artifact... repo="ghcr.io/aquasecurity/trivy-db:2"
2024-10-21T14:31:56Z ERROR [vulndb] Failed to download artifact repo="ghcr.io/aquasecurity/trivy-db:2" err="oci download error: failed to fetch the layer: GET https://ghcr.io/v2/aquasecurity/trivy-db/blobs/sha256:07994604d8e647483cf5193669d01210524ae223c43ca96364d63f59bd64d9bf: TOOMANYREQUESTS: retry-after: 665.098µs, allowed: 44000/minute"
2024-10-21T14:31:56Z FATAL Fatal error init error: DB error: failed to download vulnerability DB: OCI artifact error: failed to download vulnerability DB: failed to download artifact from any source
trivy_exitcode=3

Running Trivy in terraform/environments/observability-platform/modules/observability-platform/tenant-configuration
2024-10-21T14:31:56Z INFO [vulndb] Need to update DB
2024-10-21T14:31:56Z INFO [vulndb] Downloading vulnerability DB...
2024-10-21T14:31:56Z INFO [vulndb] Downloading artifact... repo="ghcr.io/aquasecurity/trivy-db:2"
2024-10-21T14:31:56Z ERROR [vulndb] Failed to download artifact repo="ghcr.io/aquasecurity/trivy-db:2" err="OCI repository error: 1 error occurred:\n\t* GET https://ghcr.io/v2/aquasecurity/trivy-db/manifests/2: TOOMANYREQUESTS: retry-after: 127.166µs, allowed: 44000/minute\n\n"
2024-10-21T14:31:56Z FATAL Fatal error init error: DB error: failed to download vulnerability DB: OCI artifact error: failed to download vulnerability DB: failed to download artifact from any source
trivy_exitcode=4

</details> #### `Checkov Scan` Success
<details><summary>Show Output</summary>

```hcl

*****************************

Checkov will check the following folders:
terraform/environments/observability-platform
terraform/environments/observability-platform/modules/grafana/athena-source
terraform/environments/observability-platform/modules/grafana/team
terraform/environments/observability-platform/modules/observability-platform/tenant-configuration

*****************************

Running Checkov in terraform/environments/observability-platform
Excluding the following checks: CKV_GIT_1,CKV_AWS_126,CKV2_AWS_38,CKV2_AWS_39
2024-10-21 14:31:59,204 [MainThread  ] [WARNI]  Failed to download module ministryofjustice/observability-platform-tenant/aws:1.2.0 (for external modules, the --download-external-modules flag is required)
2024-10-21 14:31:59,204 [MainThread  ] [WARNI]  Failed to download module terraform-aws-modules/lambda/aws:7.7.1 (for external modules, the --download-external-modules flag is required)
2024-10-21 14:31:59,204 [MainThread  ] [WARNI]  Failed to download module terraform-aws-modules/managed-service-prometheus/aws:2.2.3 (for external modules, the --download-external-modules flag is required)
2024-10-21 14:31:59,204 [MainThread  ] [WARNI]  Failed to download module terraform-aws-modules/managed-service-grafana/aws:2.1.1 (for external modules, the --download-external-modules flag is required)
2024-10-21 14:31:59,204 [MainThread  ] [WARNI]  Failed to download module terraform-aws-modules/iam/aws//modules/iam-policy:5.44.0 (for external modules, the --download-external-modules flag is required)
2024-10-21 14:31:59,204 [MainThread  ] [WARNI]  Failed to download module terraform-aws-modules/iam/aws//modules/iam-assumable-role:5.44.0 (for external modules, the --download-external-modules flag is required)
terraform scan results:

Passed checks: 30, Failed checks: 0, Skipped checks: 22


checkov_exitcode=0

*****************************

Running Checkov in terraform/environments/observability-platform/modules/grafana/athena-source
Excluding the following checks: CKV_GIT_1,CKV_AWS_126,CKV2_AWS_38,CKV2_AWS_39

checkov_exitcode=0

*****************************

Running Checkov in terraform/environments/observability-platform/modules/grafana/team
Excluding the following checks: CKV_GIT_1,CKV_AWS_126,CKV2_AWS_38,CKV2_AWS_39

checkov_exitcode=0

*****************************

Running Checkov in terraform/environments/observability-platform/modules/observability-platform/tenant-configuration
Excluding the following checks: CKV_GIT_1,CKV_AWS_126,CKV2_AWS_38,CKV2_AWS_39
terraform scan results:

Passed checks: 11, Failed checks: 0, Skipped checks: 4


checkov_exitcode=0

`CTFLint Scan` Success

Show Output

*****************************

Setting default tflint config...
Running tflint --init...
Installing "terraform" plugin...
Installed "terraform" (source: github.com/terraform-linters/tflint-ruleset-terraform, version: 0.9.1)
tflint will check the following folders:
terraform/environments/observability-platform
terraform/environments/observability-platform/modules/grafana/athena-source
terraform/environments/observability-platform/modules/grafana/team
terraform/environments/observability-platform/modules/observability-platform/tenant-configuration

*****************************

Running tflint in terraform/environments/observability-platform
Excluding the following checks: terraform_unused_declarations
tflint_exitcode=0

*****************************

Running tflint in terraform/environments/observability-platform/modules/grafana/athena-source
Excluding the following checks: terraform_unused_declarations
tflint_exitcode=0

*****************************

Running tflint in terraform/environments/observability-platform/modules/grafana/team
Excluding the following checks: terraform_unused_declarations
tflint_exitcode=0

*****************************

Running tflint in terraform/environments/observability-platform/modules/observability-platform/tenant-configuration
Excluding the following checks: terraform_unused_declarations
tflint_exitcode=0

`Trivy Scan` Failed

Show Output

*****************************

Trivy will check the following folders:
terraform/environments/observability-platform
terraform/environments/observability-platform/modules/grafana/athena-source
terraform/environments/observability-platform/modules/grafana/team
terraform/environments/observability-platform/modules/observability-platform/tenant-configuration

*****************************

Running Trivy in terraform/environments/observability-platform
2024-10-21T14:31:56Z	INFO	[vulndb] Need to update DB
2024-10-21T14:31:56Z	INFO	[vulndb] Downloading vulnerability DB...
2024-10-21T14:31:56Z	INFO	[vulndb] Downloading artifact...	repo="ghcr.io/aquasecurity/trivy-db:2"
2024-10-21T14:31:56Z	ERROR	[vulndb] Failed to download artifact	repo="ghcr.io/aquasecurity/trivy-db:2" err="oci download error: failed to fetch the layer: GET https://ghcr.io/v2/aquasecurity/trivy-db/blobs/sha256:07994604d8e647483cf5193669d01210524ae223c43ca96364d63f59bd64d9bf: TOOMANYREQUESTS: retry-after: 4.291µs, allowed: 44000/minute"
2024-10-21T14:31:56Z	FATAL	Fatal error	init error: DB error: failed to download vulnerability DB: OCI artifact error: failed to download vulnerability DB: failed to download artifact from any source
trivy_exitcode=1

*****************************

Running Trivy in terraform/environments/observability-platform/modules/grafana/athena-source
2024-10-21T14:31:56Z	INFO	[vulndb] Need to update DB
2024-10-21T14:31:56Z	INFO	[vulndb] Downloading vulnerability DB...
2024-10-21T14:31:56Z	INFO	[vulndb] Downloading artifact...	repo="ghcr.io/aquasecurity/trivy-db:2"
2024-10-21T14:31:56Z	ERROR	[vulndb] Failed to download artifact	repo="ghcr.io/aquasecurity/trivy-db:2" err="OCI repository error: 1 error occurred:\n\t* GET https://ghcr.io/v2/aquasecurity/trivy-db/manifests/2: TOOMANYREQUESTS: retry-after: 621.616µs, allowed: 44000/minute\n\n"
2024-10-21T14:31:56Z	FATAL	Fatal error	init error: DB error: failed to download vulnerability DB: OCI artifact error: failed to download vulnerability DB: failed to download artifact from any source
trivy_exitcode=2

*****************************

Running Trivy in terraform/environments/observability-platform/modules/grafana/team
2024-10-21T14:31:56Z	INFO	[vulndb] Need to update DB
2024-10-21T14:31:56Z	INFO	[vulndb] Downloading vulnerability DB...
2024-10-21T14:31:56Z	INFO	[vulndb] Downloading artifact...	repo="ghcr.io/aquasecurity/trivy-db:2"
2024-10-21T14:31:56Z	ERROR	[vulndb] Failed to download artifact	repo="ghcr.io/aquasecurity/trivy-db:2" err="oci download error: failed to fetch the layer: GET https://ghcr.io/v2/aquasecurity/trivy-db/blobs/sha256:07994604d8e647483cf5193669d01210524ae223c43ca96364d63f59bd64d9bf: TOOMANYREQUESTS: retry-after: 665.098µs, allowed: 44000/minute"
2024-10-21T14:31:56Z	FATAL	Fatal error	init error: DB error: failed to download vulnerability DB: OCI artifact error: failed to download vulnerability DB: failed to download artifact from any source
trivy_exitcode=3

*****************************

Running Trivy in terraform/environments/observability-platform/modules/observability-platform/tenant-configuration
2024-10-21T14:31:56Z	INFO	[vulndb] Need to update DB
2024-10-21T14:31:56Z	INFO	[vulndb] Downloading vulnerability DB...
2024-10-21T14:31:56Z	INFO	[vulndb] Downloading artifact...	repo="ghcr.io/aquasecurity/trivy-db:2"
2024-10-21T14:31:56Z	ERROR	[vulndb] Failed to download artifact	repo="ghcr.io/aquasecurity/trivy-db:2" err="OCI repository error: 1 error occurred:\n\t* GET https://ghcr.io/v2/aquasecurity/trivy-db/manifests/2: TOOMANYREQUESTS: retry-after: 127.166µs, allowed: 44000/minute\n\n"
2024-10-21T14:31:56Z	FATAL	Fatal error	init error: DB error: failed to download vulnerability DB: OCI artifact error: failed to download vulnerability DB: failed to download artifact from any source
trivy_exitcode=4

...ironments/observability-platform/modules/observability-platform/tenant-configuration/main.tf

🔧 Add Athena Data Source

1de06bb

github-actions bot added the environments-repository Used to exclude PRs from this repo in our Slack PR update label Oct 9, 2024

Gary-H9 changed the title ~~🔧 Add Athena Data Source~~ 👁️ Observability: Add Athena Data Source Oct 9, 2024

Gary-H9 added 2 commits October 9, 2024 14:13

🔧 Modify for multiple Athena Data Sources

850864d

♻️ Refactor

5342c52

Gary-H9 mentioned this pull request Oct 9, 2024

🔧 Add ability to enable Athena Data Source ministryofjustice/terraform-aws-observability-platform-tenant#52

Open

Gary-H9 marked this pull request as ready for review October 9, 2024 15:57

Gary-H9 requested review from a team as code owners October 9, 2024 15:57

Gary-H9 mentioned this pull request Oct 9, 2024

🔍 Observability: Athena Data Source ministryofjustice/analytical-platform#5382

Closed

Gary-H9 added 8 commits October 10, 2024 10:25

♻️ Refactor

19b76a4

🔧 Add try

3941aaa

📝 Attempt 1

093ae58

♻️ Refactor

5e4fcda

🔧 TEST

25feee4

🔧 TEST2

59e3628

🔧 TEST

ee5fba8

🔧 Revert to cleaner, still incorrect code

cfe8a4f

ministryofjustice deleted a comment from github-actions bot Oct 10, 2024

🔧 Another rever

4ad333e

ministryofjustice deleted a comment from github-actions bot Oct 17, 2024

🔧 plz plz plz

5eb93e3

🔧 Refactor

ae47d06

Gary-H9 added 2 commits October 21, 2024 14:55

🔧 Resolve empty tuple error

58e075e

🔧 Add nonsensitive

31182e4

julialawrence reviewed Oct 21, 2024

View reviewed changes

...ironments/observability-platform/modules/observability-platform/tenant-configuration/main.tf Outdated Show resolved Hide resolved

:

8d4ce1e

ministryofjustice deleted a comment from github-actions bot Oct 21, 2024

Gary-H9 added 3 commits October 21, 2024 15:51

🔧 for_each changes

85ff8af

🔧 There was an attempt

3d57dd7

🔧 Refactor

f3fc11f

Gary-H9 temporarily deployed to observability-platform-development October 21, 2024 16:48 — with GitHub Actions Inactive

Gary-H9 added 3 commits October 22, 2024 08:09

🔧 Remove test config

25e0411

one more time with feeling

e582e05

Panic over

e747bc6

ministryofjustice deleted a comment from github-actions bot Oct 22, 2024

Khatraf approved these changes Oct 22, 2024

View reviewed changes

Gary-H9 merged commit 6bfe015 into main Oct 22, 2024
11 of 12 checks passed

Gary-H9 deleted the obs-plat/athena-datasource-3 branch October 22, 2024 08:52

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

👁️ Observability: Add Athena Data Source #8142

👁️ Observability: Add Athena Data Source #8142

Gary-H9 commented Oct 9, 2024 •

edited

Loading

github-actions bot commented Oct 17, 2024

github-actions bot commented Oct 21, 2024

👁️ Observability: Add Athena Data Source #8142

👁️ Observability: Add Athena Data Source #8142

Conversation

Gary-H9 commented Oct 9, 2024 • edited Loading

github-actions bot commented Oct 17, 2024

Trivy Scan Success

CTFLint Scan Success

Trivy Scan Success

github-actions bot commented Oct 21, 2024

Trivy Scan Failed

CTFLint Scan Success

Trivy Scan Failed

Gary-H9 commented Oct 9, 2024 •

edited

Loading

`Trivy Scan` Success

`CTFLint Scan` Success

`Trivy Scan` Success

`Trivy Scan` Failed

`CTFLint Scan` Success

`Trivy Scan` Failed