Canary and Stable app version response header via Istio DestinationRule in subset-level traffic splitting

[kaiburjack]

We use Argo Rollouts for Canary Rollouts with the subset-level traffic splitting via Istio.
This works great, so far, until we wanted to find ways to add a custom response header indicating the version of the application that either the stable or canary DestinationRule was serving. We use header-based routing (not weighted routing) to the canary version, such that e2e/acceptance tests can happen against the canary without affecting what end-users see, until the e2e tests succeed and from there on weight-based traffic shifting takes place.

However, testers would like to know when they are using the designated canary header that they indeed are hitting the canary version of the service (and which version exactly, given the high frequency of canary deployments that happen currently). We thought about implementing this via Istio's DestinationRule headers.response.add mechanism. However, that of course poses a problem since one particular rollout (via Argo CD + Helm) only knows the current (new) service version when applied/synced to the cluster. And the Argo Rollouts controller is then responsible for creating a new ReplicaSet for the new version while also having the ReplicaSet for the old version around.

Now, since at deploy (Helm chart release) time, we don't know what the old service version was, there could be some kind of mechanism in Argo Rollouts itself to maybe also augment a DestinationRule with a headers.response.add field to allow application/deployment authors to add headers based on the new version of the service which is currently rolled-out using the canary, while preserving the header value for the old version?

I just wanted to know what options we would have here. Thanks!

I mean, one solution would obviously be to let the application/service itself respond with such a "I am version XYZ" header, but a cross-cutting solution with Istio would be preferable, also given that we have a common Helm chart for application rollouts, which is reused for multiple services, and we could add a X-Service-Version header perfectly based on Helm's {{ .Release.Name }}, for example, on the Argo Rollouts Rollout resource.

[kaiburjack]

Alright, we will be going with a custom nginx as a sidecar in the application's pod to augment responses with a "X-Service-Version" response header to tell the application/service version. That's the simplest solution as it can be implemented as part of the application's/service's Helm chart.