[Release-1.27] [Backport] node-ip rke2 flag isn't always passed to kubelet

[aganesh-suse]

Release 1.27 - Backport issue tracking for: #4759

[fmoral2]

Validated on Version:

-$ rke2 version v1.27.7-rc2+rke2r1 (5fdc8d79f890391e6ce3485a5b92fa4925398210)

Environment Details

Infrastructure
Cloud EC2 instance

Node(s) CPU architecture, OS, and Version:
PRETTY_NAME="Ubuntu 22.04.1 LTS"
NAME="Ubuntu"
VERSION_ID="22.04"

Cluster Configuration:
1 node servers

Steps to validate the fix

1. Install rke2 in default config passing node ip as arg
2. Check kubelet logs args to validate the default node-ip is there
3. Validate nodes and pods are running and ok
4. Install rke2 not passing node ip as arg
5. Check kubelet logs args to validate the default node-ip is there
6. Validate nodes and pods are running ok

Validation Results:

------without  node ip 

$ cat /etc/rancher/rke2/config.yaml 
write-kubeconfig-mode: "0644"
tls-san:
  - fake.fqdn.value





$ rke2 -v
rke2 version v1.27.7-rc2+rke2r1 (5fdc8d79f890391e6ce3485a5b92fa4925398210)
go version go1.20.10 X:boringcrypto




~$ sudo journalctl -xeu rke2-server.service | grep 'Running kubelet'
Oct 25 09:41:35 ip-172-31-0-10 rke2[1743]: time="2023-10-25T09:41:35Z" level=info msg="Running kubelet --address=0.0.0.0 --alsologtostderr=false --anonymous-auth=false --authentication-token-webhook=true --authorization-mode=Webhook --cgroup-driver=systemd --client-ca-file=/var/lib/rancher/rke2/agent/client-ca.crt --cloud-provider=external --cluster-dns=10.43.0.10 --cluster-domain=cluster.local --container-runtime-endpoint=unix:///run/k3s/containerd/containerd.sock --containerd=/run/k3s/containerd/containerd.sock --eviction-hard=imagefs.available<5%,nodefs.available<5% --eviction-minimum-reclaim=imagefs.available=10%,nodefs.available=10% --fail-swap-on=false --feature-gates=CloudDualStackNodeIPs=true --healthz-bind-address=127.0.0.1 --hostname-override=ip-172-31-0-10.us-east-2.compute.internal --kubeconfig=/var/lib/rancher/rke2/agent/kubelet.kubeconfig --log-file=/var/lib/rancher/rke2/agent/logs/kubelet.log --log-file-max-size=50 --logtostderr=false --node-ip=172.31.0.10,2600:1f16:1d38:1c00:fb2c:a16a:6a33:c14f --node-labels= --pod-infra-container-image=index.docker.io/rancher/mirrored-pause:3.6 --pod-manifest-path=/var/lib/rancher/rke2/agent/pod-manifests --read-only-port=0 --resolv-conf=/run/systemd/resolve/resolv.conf --serialize-image-pulls=false --stderrthreshold=FATAL --tls-cert-file=/var/lib/rancher/rke2/agent/serving-kubelet.crt --tls-private-key-file=/var/lib/rancher/rke2/agent/serving-kubelet.key"
    

--------       --node-ip=172.31.0.10,        ---------------

$ k get nodes -o wide
NAME                                        STATUS   ROLES                       AGE     VERSION          INTERNAL-IP   EXTERNAL-IP    OS-IMAGE             KERNEL-VERSION    CONTAINER-RUNTIME
ip-172-31-0-10.us-east-2.compute.internal   Ready    control-plane,etcd,master   5m29s   v1.27.7+rke2r1   172.31.0.10   52.15.70.164   Ubuntu 22.04.1 LTS   5.15.0-1019-aws   containerd://1.7.7-k3s1



~$ k get pods -A
NAMESPACE     NAME                                                                 READY   STATUS      RESTARTS   AGE
kube-system   cloud-controller-manager-ip-172-31-0-10.us-east-2.compute.internal   1/1     Running     0          5m21s
kube-system   etcd-ip-172-31-0-10.us-east-2.compute.internal                       1/1     Running     0          5m5s
kube-system   helm-install-rke2-canal-dwfqp                                        0/1     Completed   0          5m15s
kube-system   helm-install-rke2-coredns-m7lbg                                      0/1     Completed   0          5m15s
kube-system   helm-install-rke2-ingress-nginx-t4fbd                                0/1     Completed   0          5m15s
kube-system   helm-install-rke2-metrics-server-mj9hb                               0/1     Completed   0          5m15s
kube-system   helm-install-rke2-snapshot-controller-crd-2fdzr                      0/1     Completed   0          5m15s
kube-system   helm-install-rke2-snapshot-controller-qlckd                          0/1     Completed   1          5m15s
kube-system   helm-install-rke2-snapshot-validation-webhook-jtklz                  0/1     Completed   0          5m15s
kube-system   kube-apiserver-ip-172-31-0-10.us-east-2.compute.internal             1/1     Running     0          4m46s
kube-system   kube-controller-manager-ip-172-31-0-10.us-east-2.compute.internal    1/1     Running     0          5m23s
kube-system   kube-proxy-ip-172-31-0-10.us-east-2.compute.internal                 1/1     Running     0          5m16s
kube-system   kube-scheduler-ip-172-31-0-10.us-east-2.compute.internal             1/1     Running     0          5m23s
kube-system   rke2-canal-qfjst                                                     2/2     Running     0          4m54s
kube-system   rke2-coredns-rke2-coredns-autoscaler-6f97df447-zlcd8                 1/1     Running     0          5m4s
kube-system   rke2-coredns-rke2-coredns-f6c9f9649-d5mrl                            1/1     Running     0          5m4s
kube-system   rke2-ingress-nginx-controller-c8f5t                                  1/1     Running     0          3m37s
kube-system   rke2-metrics-server-6d79d977db-t4j54                                 1/1     Running     0          4m2s
kube-system   rke2-snapshot-controller-7d6476d7cb-hkrnp                            1/1     Running     0          3m46s
kube-system   rke2-snapshot-validation-webhook-5649fbd66c-7vc2k                    1/1     Running     0          3m47s





#########   with node ip   ###########


~$ cat /etc/rancher/rke2/config.yaml 
write-kubeconfig-mode: "0644"
tls-san:
  - fake.fqdn.value
node-ip: 172.31.0.10


$ rke2 -v
rke2 version v1.27.7-rc2+rke2r1 (5fdc8d79f890391e6ce3485a5b92fa4925398210)
go version go1.20.10 X:boringcrypto


:~$ sudo journalctl -xeu rke2-server.service | grep 'Running kubelet' 

Oct 25 09:49:10 ip-172-31-0-10 rke2[15365]: time="2023-10-25T09:49:10Z" level=info msg="Running kubelet --address=0.0.0.0 --alsologtostderr=false --anonymous-auth=false --authentication-token-webhook=true --authorization-mode=Webhook --cgroup-driver=systemd --client-ca-file=/var/lib/rancher/rke2/agent/client-ca.crt --cloud-provider=external --cluster-dns=10.43.0.10 --cluster-domain=cluster.local --container-runtime-endpoint=unix:///run/k3s/containerd/containerd.sock --containerd=/run/k3s/containerd/containerd.sock --eviction-hard=imagefs.available<5%,nodefs.available<5% --eviction-minimum-reclaim=imagefs.available=10%,nodefs.available=10% --fail-swap-on=false --feature-gates=CloudDualStackNodeIPs=true --healthz-bind-address=127.0.0.1 --hostname-override=ip-172-31-0-10 --kubeconfig=/var/lib/rancher/rke2/agent/kubelet.kubeconfig --log-file=/var/lib/rancher/rke2/agent/logs/kubelet.log --log-file-max-size=50 --logtostderr=false --node-ip=172.31.0.10 --node-labels= --pod-infra-container-image=index.docker.io/rancher/mirrored-pause:3.6 --pod-manifest-path=/var/lib/rancher/rke2/agent/pod-manifests --read-only-port=0 --resolv-conf=/run/systemd/resolve/resolv.conf --serialize-image-pulls=false --stderrthreshold=FATAL --tls-cert-file=/var/lib/rancher/rke2/agent/serving-kubelet.crt --tls-private-key-file=/var/lib/rancher/rke2/agent/serving-kubelet.key"



--------     --node-ip=172.31.0.10     -----------