Availability zone aware scaling

[andrii-korotkov-verkada]

Proposal

An ability to easily configure availability zone aware scaling, bringing up more pods in zones which are loaded more.

For example, suppose the target cpu utilization is 80%, while pods in availability zone a (AWS) have 90% and pods in other availability zones have 70%. Thus an average value would be below 80% assuming even split of pods between availability zones. The feature would allow to identify that az a needs more pods and schedule them.

This may need creation of multiple hpa objects and for hpa to support a filter on availability zone.

For custom metrics like Cloudwatch it may require a template for values, e.g. for dimension AvailabilityZone specify {{topology.kubernetes.io/zone}}.

Use-Case

Sometimes availability zones get uneven traffic, causing pods in a specific az being overloaded. Default approach of HPA and KEDA may not address this, since average metrics may look okay. A solution I could find involve create separate deployments and hpa/keda for each availability zone, but that's a lot of configuration considering multiple AWS regions, even while using Kustomize for Kubernetes configs. Simple configuration would help to adopt the feature.

Is this a feature you are interested in implementing yourself?

No

Anything else?

No response

[JorTurFer]

Hello!
I'm not totally sure about if I have understood the requirement, let me check it
You are deploying a workload scaled by KEDA which works on an autoscaling group with MultiAz , and you would like to have a mechanism to scale the workload based on the load of the az instead of based on all the workload across all the az, right?

If this is the case, I'm not sure if it's something doable by KEDA as KEDA relies on the HPA Controller, and AFAIK it doesn't support this "smart" logic sharing the workload and the HPA. I mean, you have 1 workload and 1 HPA, so from scaling pov, if you need an extra instance, it's the scheduler who decides where it has to be scheduled. If you want to achieve this behavior, you will need to split the workload into a workload per az and scaling them one by one.

Maybe I'm missing some point and this is doable. WDTY @zroubalik ?

[andrii-korotkov-verkada]

Hello!
Yes, you understand the goals correctly. My hope was to come up with some mechanism which reduces configuration amount, e.g. maybe 1 KEDA can create mutliple HPA per each availability zone, assuming HPA can have a filter on which availability zone it's working on, all while keeping one deployment/rollout configuration. Or maybe there's a way for Kubernetes to support one deployment configuration, which creates multiple objects per availability zone under the hood, though that's not a question to KEDA.

[tomkerkhove]

I fully get the goal, but this seems more like a scheduling problem rather than scaling problem. KEDA scales, for example, deployments by telling it to scale out but it is up to the scheduler to decide on what node to start a new pod.

Even if we create multiple HPAs or ScaledObjects (which is not good), we cannot achieve this goal.

I think using topology spread constraints is the best approach for now