Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

fix: truncate label values longer than 63 characters #6382

Open
wants to merge 2 commits into
base: main
Choose a base branch
from

Conversation

omerap12
Copy link
Contributor

Provide a description of what has been changed
When creating a ScaledObject with a name exceeding 63 characters, an error occurs due to Kubernetes' label length limitations. This PR addresses the issue by truncating the ScaledObject name to ensure it adheres to the label length requirements.

Checklist

Fixes #6370

Relates to #

Made local testing:
deployment.yaml:

apiVersion: apps/v1
kind: Deployment
metadata:
  name: nginx
  labels:
    app: nginx
spec:
  replicas: 3
  selector:
    matchLabels:
      app: nginx
  template:
    metadata:
      labels:
        app: nginx
    spec:
      containers:
      - name: nginx
        image: nginx:1.14.2
        ports:
        - containerPort: 80
        resources:
          requests:
            cpu: "100m"
            memory: "128Mi"
          limits:
            cpu: "200m"
            memory: "256Mi"

scaledobject.yaml:

apiVersion: keda.sh/v1alpha1
kind: ScaledObject
metadata:
  name: this-is-64-characters-name-xxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxx
spec:
  minReplicaCount: 1
  maxReplicaCount: 10
  scaleTargetRef:
    name: nginx
  triggers:
  - type: cpu
    metricType: Utilization
    metadata:
      type: Utilization
      value: "60"

and get:

 ~/ k get scaledobjects.keda.sh
NAME                                                               SCALETARGETKIND      SCALETARGETNAME   MIN   MAX   READY   ACTIVE   FALLBACK   PAUSED    TRIGGERS   AUTHENTICATIONS   AGE
this-is-64-characters-name-xxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxx   apps/v1.Deployment   nginx             1     10    True    True     False      Unknown                                6m28s

and:

Name:         this-is-64-characters-name-xxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxx
Namespace:    default
Labels:       scaledobject.keda.sh/name=this-is-64-characters-name-xxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxx
Annotations:  <none>
API Version:  keda.sh/v1alpha1
Kind:         ScaledObject
Metadata:
  Creation Timestamp:  2024-11-30T15:54:10Z
  Finalizers:
    finalizer.keda.sh
  Generation:        1
  Resource Version:  4932
  UID:               e6447cc3-14ec-45bf-a3dd-0fd8e4267743
Spec:
  Max Replica Count:  10
  Min Replica Count:  1
  Scale Target Ref:
    Name:  nginx
  Triggers:
    Metadata:
      Type:       Utilization
      Value:      60
    Metric Type:  Utilization
    Type:         cpu
Status:
  Conditions:
    Message:               ScaledObject is defined correctly and is ready for scaling
    Reason:                ScaledObjectReady
    Status:                True
    Type:                  Ready
    Message:               Scaling is performed because triggers are active
    Reason:                ScalerActive
    Status:                True
    Type:                  Active
    Message:               No fallbacks are active on this scaled object
    Reason:                NoFallbackFound
    Status:                False
    Type:                  Fallback
    Status:                Unknown
    Type:                  Paused
  Hpa Name:                keda-hpa-this-is-64-characters-name-xxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxx
  Last Active Time:        2024-11-30T16:00:22Z
  Original Replica Count:  3
  Resource Metric Names:
    cpu
  Scale Target GVKR:
    Group:            apps
    Kind:             Deployment
    Resource:         deployments
    Version:          v1
  Scale Target Kind:  apps/v1.Deployment
Events:
  Type    Reason              Age    From           Message
  ----    ------              ----   ----           -------
  Normal  KEDAScalersStarted  6m29s  keda-operator  Scaler cpu is built.
  Normal  KEDAScalersStarted  6m29s  keda-operator  Started scalers watch
  Normal  ScaledObjectReady   6m29s  keda-operator  ScaledObject is ready for scaling

@omerap12 omerap12 requested a review from a team as a code owner November 30, 2024 16:01
@@ -293,7 +287,7 @@ func (r *ScaledObjectReconciler) getScaledObjectMetricSpecs(ctx context.Context,
Metric: autoscalingv2.MetricIdentifier{
Name: compMetricName,
Selector: &metav1.LabelSelector{
MatchLabels: map[string]string{kedav1alpha1.ScaledObjectOwnerAnnotation: scaledObject.Name},
MatchLabels: map[string]string{kedav1alpha1.ScaledObjectOwnerAnnotation: scaledObjectName},
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I'm not 100% sure if we should do this or just print an error. We use this label in to recover the SO name during the HPA querying and I'd say that the querying will fail if we truncate this name. But maybe I'm wrong and if we cover the scenario with and e2e test to cover it, I'm fine 😄

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@kedacore/keda-contributors

Copy link
Member

@wozniakjan wozniakjan Dec 2, 2024

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

oddly enough, HPA seems to have the label truncated since 2021, there it's probably not used that much.

For sure we should add e2e tests for trunc name cases, I'd be curious to see what is the behaviour when two SO with names 63+ chars that differ only in the last truncated characters.

Alternatively, can we refactor the code to use SO uid field? that is guaranteed to be within the bounds but not sure if that would be a breaking change. Or just document that SOs with more than 63 chars are not allowed :)

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

oddly enough, HPA seems to have the label #1631, there it's probably not used that much.

That applies to the HPA name, and this label is this other line -> https://github.com/kedacore/keda/pull/1631/files#diff-a43846d8ca384145be0f11e006bc0ce798b50c992df46e9b13ff0c55c290bae0L157

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The HPA name isn't relevant if there isn't more HPAs in conflict in the namespace, but the ScaledObject name is used to get the SO

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

So, do we move forward by simply printing an error and relying on an end-to-end job to cover this?

Copy link
Member

@wozniakjan wozniakjan Dec 4, 2024

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

as @JorTurFer mentioned, the ScaledObject will be patched with a truncated label now but getting metrics may probably not work because of how it's implemented in the provider.go. I don't have any good ideas how to work around that unless we refactor the internals to not use the name but uid for example.

perhaps a good start is to create an e2e test covering the case of ScaledObject with a long name so we can see if our assumption of a failure is correct or no. To be extra safe, we may want to also add an e2e test where two ScaledObjects in the same namespace have the same truncated name.

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I like @wozniakjan's suggestion

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

Fail to create HPA if ScaledObject name is longer than 63 characters
4 participants