Notify downstream consumers of new AWS Inspector scan findings

[mogul]

User Story

In order to inform downstream GSA consumers when an EKS instance has new security findings, the data.gov team wants interested parties to receive an email notifying them of the new findings.

Acceptance Criteria

[ACs should be clearly demoable/verifiable whenever possible. Try specifying them using BDD.]

• GIVEN I have a list of email addresses/Slack channels
  WHEN new findings for the EC2 and ECR subjects that AWS Inspector watches are ingested into our logging solution
  THEN the list of email addresses/Slack channels receive a notification with a summary of the things that AWS Inspector scans for
  AND the notification includes a list of new findings
  AND the notification includes a list of not-new findings that still exist.

Background

[Any helpful contextual notes or links to artifacts/evidence, if needed]

Security Considerations (required)

This work will make it easier to ensure that interested parties (eg the data.gov team, GSA IT) find out about security problems as they're noted in a provisioned EKS instance and the containers in the related ECR instance, without having to go look for them in the AWS Console dashboard or manually watch logs.

Sketch

Ideally this will not by done by following this AWS EventBridge example, though it's a fallback option.

Instead we want logs drained to an S3 bucket as described in #3753. The contents of that bucket are then ingested into a central logging/monitoring/alerting solution (eg New Relic Logs) alongside cloud.gov-app-generated logs. Notifications are generated when the monitored logs trigger alerts, as described in #3062.

[nickumia-reisys]

I think this is the best issue to document this on, so here it goes..

Following the official AWS Inspector user guide, I was able to create an S3 Bucket to export findings from Inspector. The following steps provide a summary and highlight key obstacles:

• Create KMS key for S3 Encryption with policy allowing Inspector to access
  • Normal Symmetric KMS key
  • Ensure Key is created in the correct region
• Create S3 Bucket with the same key
  • The bucket does not need public access
  • Ensure bucket references the correct region (even though it's a global service)
• Add/modify S3 Bucket policy allowing Inspector to perform required actions
• Go to AWS Inspector Findings page,
  • Click on Export findings
  • Modify filter to be Show all instead of Active
  • Select either json or csv
  • Enter the S3 URI (in this case, s3://datagov-inspector-logs)
    • Note: If a specific .csv extension isn't provided, the URI will just be a prefix with Inspector creating a random name for the log file.
  • Select the KMS Key created above.
  • Click on Export Button
• Use AWS S3 CLI to download the results,
  • aws s3 ls s3://$BUCKET_NAME/
  • aws s3 cp s3://$BUCKET_NAME/

KMS Key Policy

{
    "Version": "2012-10-17",
    "Id": "key-consolepolicy-3",
    "Statement": [
        {
            "Sid": "Enable IAM User Permissions (whoever needs to use the key)",
            "Effect": "Allow",
            "Principal": {
                "AWS": [
                    "arn:aws:iam::<account-id>:<user/role>"
                ]
            },
            "Action": "kms:*",
            "Resource": "*"
        },
        {
            "Sid": "Allow inspector to perform kms actions",
            "Effect": "Allow",
            "Principal": {
                "Service": "inspector2.amazonaws.com"
            },
            "Action": [
                "kms:Decrypt",
                "kms:GenerateDataKey*"
            ],
            "Resource": "*",
            "Condition": {
                "StringEquals": {
                    "aws:SourceAccount": "<account-id>"
                },
                "ArnLike": {
                    "aws:SourceArn": "arn:aws:inspector2:<region>:<account-id>:report/*"
                }
            }
        }
    ]
}

S3 Bucket Policy

{
    "Version": "2012-10-17",
    "Statement": [
        {
            "Sid": "Allow inspector to perform Put and Delete actions on s3",
            "Effect": "Allow",
            "Principal": {
                "Service": "inspector2.amazonaws.com"
            },
            "Action": [
                "s3:PutObject",
                "s3:PutObjectAcl",
                "s3:AbortMultipartUpload"
            ],
            "Resource": "arn:aws:s3:::datagov-inspector-logs/*",
            "Condition": {
                "StringEquals": {
                    "aws:SourceAccount": "<account-id>"
                },
                "ArnLike": {
                    "aws:SourceArn": "arn:aws:inspector2:<region>:<account-id>:report/*"
                }
            }
        },
        {
            "Sid": "Statement1",
            "Effect": "Allow",
            "Principal": {
                "AWS": [
                    "arn:aws:iam::<account-id>:<user/role>"
                ]
            },
            "Action": "s3:*",
            "Resource": "arn:aws:s3:::<bucket-name>/*"
        }
    ]
}

[nickumia-reisys]

Note that the above setup was a one-time process. From now on, users should be able to a navigate to the Inspector Findings page and just use the bucket and kms key that already exists. This is a manual process, so it doesn't solve this issue. It was just a workaround to get results out in a temporary fashion. This issue is still about automating the notification of new findings and reports.

Note: if users are not in the Bucket and Key policies, they will not be able to download the results.

[nickumia-reisys]

The docs do mention a way of notifying you of AWS Inspector Findings which involves setting up an SNS topic with EventBridge which sends out emails.

[mogul]

If it's possible, when you automate this, please make it possible to specify the bucket. I want to be able to direct backups, etc to the same bucket, and then also use it for restores in other clusters, so it's preferable that the bucket lifecycle not be tied to the EKS instance lifecycle.