Add support for archival

[alexandrevilain]

This PR closes #364

[yujunz]

@alexandrevilain do you have an ETA for archival support? It is mandatory for our production deployment.

Alternatively, we can implement #418 but I guess that will take longer.

[alexandrevilain]

Hi @yujunz ! Happy to read that you’re deploying this project in production!

Even if we are using this operator in production at work, it’s not my daily job to maintain it.

So I’m sorry but I can’t give you any ETA. Maybe during the following weeks, but not 100% confident about that.

[yujunz]

Understood. We just need to make decisions on either turning to helm chart for flexibility and features completeness or sticking to operator to reduce maintenance overhead in the long term.

A few weeks fit our schedule well. Let me know if anything I can help to accelerate the delivery.

[alexandrevilain]

@yujunz Are you using EKS ? I'll need some testers to check if IAM authentication will work for archival :)

[yujunz]

@yujunz Are you using EKS ? I'll need some testers to check if IAM authentication will work for archival :)

Yeah, glad to help. Let me know when it is ready for testing.

[DodgeCamaro]

@alexandrevilain I can help you with test IAM at AWS EKS

[alexandrevilain]

Hi @yujunz and @DodgeCamaro !
I’ve tried this feature with OVHCloud’s s3 works like a charm!
If you have time could you please try it with EKS + IAM role?
If it works could you please provide me a role example so I can provide it in the documentation? Thanks!

[DodgeCamaro]

example of trust_relationships

{
    "Version": "2012-10-17",
    "Statement": [
        {
            "Effect": "Allow",
            "Principal": {
                "Federated": "arn:aws:iam::<account_id>:oidc-provider/oidc.eks.<aws_region>.amazonaws.com/id/<cluster_id>"
            },
            "Action": "sts:AssumeRoleWithWebIdentity",
            "Condition": {
                "StringEquals": {
                    "oidc.eks.<aws_region>.amazonaws.com/id/<cluster_id>:sub": "system:serviceaccount:temporal:temporal-frontend"
                }
            }
        }
    ]
}

s3 policy example

{
    "Version": "2012-10-17",
    "Statement": [
        {
            "Effect": "Allow",
            "Principal": {
                "AWS": "arn:aws:iam::<account_id>:role/<iam_role>"
            },
            "Action": [
                "s3:PutObjectAcl",
                "s3:PutObject",
                "s3:GetObjectVersion",
                "s3:GetObject",
                "s3:DeleteObject"
            ],
            "Resource": "arn:aws:s3:::<bucket_name>/*"
        },
        {
            "Effect": "Allow",
            "Principal": {
                "AWS": "arn:aws:iam::<account_id>:role/<iam_role>"
            },
            "Action": "s3:ListBucket",
            "Resource": "arn:aws:s3:::<bucket_name>"
        }
    ]
}

[DodgeCamaro]

Update example at this PR #445
Currently, I've testing archival and waiting when retention period move my workflow to archival

UPD: At k8s serviceAccount have same Annotations
eks.amazonaws.com/role-arn=arn:aws:iam::<account-id>:role/temporal-frontend-role
It won't work, because trust_relationships have conditions only from namespace - temporal, and SA temporal-frontend

[alexandrevilain]

It won't work, because trust_relationships have conditions only from namespace - temporal, and SA temporal-frontend

I don't understand. What's missing on the operator side ? A service account per service type ?

[DodgeCamaro]

All temporal services have their own ServiceAccount.
It can be fixed by adding SA to IAM Role trust_relationships
Example for add to doc

{
    "Version": "2012-10-17",
    "Statement": [
        {
            "Effect": "Allow",
            "Principal": {
                "Federated": "arn:aws:iam::<account_id>:oidc-provider/oidc.eks.<aws_region>.amazonaws.com/id/<cluster_id>"
            },
            "Action": "sts:AssumeRoleWithWebIdentity",
            "Condition": {
                "StringEquals": {
                    "oidc.eks.<aws_region>.amazonaws.com/id/<cluster_id>:sub": ["system:serviceaccount:<temporal_ns>:<temporal_sa>"]
                }
            }
        }
    ]
}

[alexandrevilain]

Alright, so with both iam role on ServiceAccount + trust relationship it works well ? No need to update the operator code ?

[DodgeCamaro]

Alright, so with both iam role on ServiceAccount + trust relationship it works well ? No need to update the operator code ?

Don't need any update from operator code

[alexandrevilain]

Thanks, last test step for me is GCS

[sonarqubecloud]

Kudos, SonarCloud Quality Gate passed!   

0 Bugs
0 Vulnerabilities
0 Security Hotspots
2 Code Smells

No Coverage information
0.0% Duplication