Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

With humanize-memory some MEM recommendation use "millibyte" unit #7770

Open
kyleli666 opened this issue Jan 26, 2025 · 3 comments
Open

With humanize-memory some MEM recommendation use "millibyte" unit #7770

kyleli666 opened this issue Jan 26, 2025 · 3 comments
Assignees
Labels
area/vertical-pod-autoscaler kind/bug Categorizes issue or PR as related to a bug.

Comments

@kyleli666
Copy link

Which component are you using?:
/area vertical-pod-autoscaler

What version of the component are you using?:

Component version: Image tag 1.3.0

What k8s version are you using (kubectl version)?:

kubectl version Output
$ kubectl version
Client Version: v1.29.13
Kustomize Version: v5.0.4-0.20230601165947-6ce0bf390ce3
Server Version: v1.30.8-eks-2d5f260

What environment is this in?:
EKS

What did you expect to happen?:
Memory recommendation in unit "Mi"

What happened instead?:
Memory recommendation in unit "m"

How to reproduce it (as minimally and precisely as possible):
my vpa is installed with

tag=1.3.0; helm upgrade -i -n kube-system vpa fairwinds-stable/vpa -f vpa-values.yaml --set recommender.image.tag=$tag,updater.image.tag=$tag,admissionController.image.tag=$tag
# vpa-values.yaml
recommender:
  # recommender.enabled -- If true, the vpa recommender component will be installed.
  enabled: true
  # recommender.extraArgs -- A set of key-value flags to be passed to the recommender
  extraArgs:
    v: "4"
    humanize-memory: true # starting from 1.3.0
    pod-recommendation-min-cpu-millicores: 10
    pod-recommendation-min-memory-mb: 50
    target-cpu-percentile: 0.50
    target-memory-percentile: 0.50
  replicaCount: 1

Without "humanize-memory: true", I get this result

$ k get vpa -A -w
NAMESPACE     NAME                       MODE      CPU    MEM         PROVIDED   AGE
kube-system   vpa-admission-controller   Auto      11m    52428800    True       158d
kube-system   vpa-recommender            Auto      11m    63544758    True       158d
kube-system   vpa-updater                Auto      11m    78221997    True       158d
logging       fluent-bit-service         Initial   126m   225384266   True       3d2h
monitoring    grafana                    Auto      11m    93633096    True       146d

With "humanize-memory: true", I get a result where 50Mi looks good but the others not.

$ k get vpa -A -w
NAMESPACE     NAME                       MODE      CPU    MEM             PROVIDED   AGE
kube-system   vpa-admission-controller   Auto      11m    50Mi            True       158d
kube-system   vpa-recommender            Auto      11m    63543705600m    True       158d
kube-system   vpa-updater                Auto      11m    93637836800m    True       158d
logging       fluent-bit-service         Initial   126m   272063528960m   True       3d2h
monitoring    grafana                    Auto      11m    93637836800m    True       146d

My VPA Objects for the VPA components.

---
apiVersion: autoscaling.k8s.io/v1
kind: VerticalPodAutoscaler
metadata:
  name: vpa-admission-controller
  namespace: kube-system
spec:
  targetRef:
    apiVersion: apps/v1
    kind: Deployment
    name: vpa-admission-controller
  updatePolicy:
    updateMode: Auto
---
apiVersion: autoscaling.k8s.io/v1
kind: VerticalPodAutoscaler
metadata:
  name: vpa-recommender
  namespace: kube-system
spec:
  targetRef:
    apiVersion: apps/v1
    kind: Deployment
    name: vpa-recommender
  updatePolicy:
    updateMode: Auto
  resourcePolicy:
    containerPolicies:
      - containerName: '*'
        minAllowed:
          memory: 50Mi
---
apiVersion: autoscaling.k8s.io/v1
kind: VerticalPodAutoscaler
metadata:
  name: vpa-updater
  namespace: kube-system
spec:
  targetRef:
    apiVersion: apps/v1
    kind: Deployment
    name: vpa-updater
  updatePolicy:
    updateMode: Auto

Anything else we need to know?:
my logs recommender.log

@kyleli666 kyleli666 added the kind/bug Categorizes issue or PR as related to a bug. label Jan 26, 2025
@adrianmoisey
Copy link
Member

cc @omerap12

@kyleli666 kyleli666 changed the title With humanize-memory some MEM recommendation use "milibyte" unit With humanize-memory some MEM recommendation use "millibyte" unit Jan 26, 2025
adrianmoisey added a commit to adrianmoisey/autoscaler that referenced this issue Jan 26, 2025
@adrianmoisey
Copy link
Member

I was debugging this. I've made a failing test to assist with fixing it: #7771
At the moment my best guess is that this line is returning it to a non-humanised version:

quantity = resource.MustParse(humanizedValue)

@omerap12
Copy link
Member

Thanks for sharing this. Ill take a look.
/assign

adrianmoisey added a commit to adrianmoisey/autoscaler that referenced this issue Jan 26, 2025
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
area/vertical-pod-autoscaler kind/bug Categorizes issue or PR as related to a bug.
Projects
None yet
Development

No branches or pull requests

4 participants