Investigate IAM Role-Based Permissions for External S3 Buckets

[anayeaye]

What

The VEDA dynamic tiler needs to stream COG data from externally hosted protected S3 buckets. The work recorded in this ticket spins off the primary earthdata login solution for accessing data in protected DAAC buckets (#25). It turns out there are many in-house examples of using the Secure Token Service to obtain temporary role-based credentials (thank you @alukach, @sharkinsspatial), so this 'investigation' is more of a compilation of resources and a record of the UAH deployment we used to prove the concept for the delta-backend.

How

We demonstrated that a policy attached to an S3 bucket in an external AWS account that permits the GetObject action for a specific VEDA data access role can be used to allow the pgstac-titiler raster-api in the veda-backend to stream externally hosted protected data. In the raster API settings, an sts client is used to get AWS session credentials for a role and these sessions are passed into the gdal configuration for the titiler mosaic and COG factories (this proof of concept work is committed in the draft edl-4-raster-api PR #56).

Data access role requirements

Trust policy must allow sts:AssumeRole for root AWS root Principle in the VEDA CDK stack's host account. Without this policy in the data access role, the the sts client will be unable to obtain the data access role's credentials because lambda execution role needs permission to assume the role.

Read permissions

In the host account, permissions must be added for the data access role to read the data bucket (this is also required for the role to stream data from protected buckets within the VEDA stack's host account).

{
  "Version": "2012-10-17",
  "Statement": [
    {
      "Effect": "Allow",
      "Principal": {
        "AWS": "arn:aws:iam::<veda acct>:role/<data-access-role>"
      },
      "Action": ["s3:GetObject"],
      "Resource": ["arn:aws:s3:::<daac protected hls bucket>/*"]
    }
  ]
}

STS assume role

The Security Token Service dispenses temporary credentials for a given user or role. Some examples of projects that use sts client to obtain credentials: cmr-pgstac-loader, veda-stac-ingestion-registry

DAAC permissions request

Draft google document with material for a request for role based access to DAAC protected buckets.

Related #25, #55, #58, WIP PR #56