Why does Argo Rollouts require cluster-scoped read operations on Secrets?

[agrawal93]

Summary

What do you want to know about this project?
Argo Rollouts Operator throws the error below, if I revoke all access to Secrets in ClusterRoleBinding. I would like to understand why does Argo Rollouts Operator need privileges to read list of secrets at cluster scope.

E1119 12:13:09.987199       1 reflector.go:153] pkg/mod/k8s.io/[email protected]/tools/cache/reflector.go:105: Failed to list *v1.Secret: secrets is forbidden: User "system:serviceaccount:argo-rollouts-stage:argo-rollouts" cannot list resource "secrets" in API group "" at the cluster scope

Motivation

Why do you need to know this, any examples or use cases you could include?
As a central Kubernetes management team in an organization, we would like to give restricted access for Secrets, and just wanted to get an idea around what's the use-case which the operator is supporting.

[alexmt]

@agrawal93 , secret read is required because secret can be referenced from AnalysysTemplates and AnalysysRun.

[agrawal93]

AnalysisTemplate and AnalysisRun are at namespace level, so in that case shouldn't the read for secrets be namespace scoped?

[jessesuen]

@agrawal93 The rollout controller needs to operate on an AnalysisRun in any namespace. AnalysisRuns are namespaced-scoped resources, which can reference a Secret.

spec:
  args:
  - name: api-token
    valueFrom:
      secretKeyRef:
        name: token-secret
        key: apiToken

The valueFrom secret referenece is primarily intended to be used by the web metric, in order to include do things like authenticated queries. Therefore in order for the Rollout controller to make authenticated queries that require these secret, it needs cluster scoped access to query K8s for the Secret values.

NOTE: if you do not need this feature, you should be able to remove the secret access from the controller's ClusterRole, and only give a Role (in the argo-rollouts namespace) which has namespace access to secrets. The namespace-scoped access to secrets in the argo-rollouts namespace is necessary because the tokens to Wavefront, DataDog, NewRelic are all stored in the argo-rollouts namespace.