Validate kube-cloud-controller-manager-arg new arg pulled in from k3s

[rancher-max]

Defaults still appear to come from k3s: https://github.com/k3s-io/k3s/blob/6aef26e94b57934185eb2e184ec75ce77dc5ca2b/pkg/daemons/control/server.go#L302-L317. We should make sure this arg is working in rke2 as expected by changing multiple of these.

[fmoral2]

Validated on Version:

-$ rke2 version v1.28.3-rc2+rke2r1 (0d0d0e4879fdf95254461e3a49224f75d7b2dc3d)

Environment Details

Infrastructure
Cloud EC2 instance

Node(s) CPU architecture, OS, and Version:
PRETTY_NAME="Ubuntu 22.04.1 LTS"
NAME="Ubuntu"
VERSION_ID="22.04"

Cluster Configuration:
2 node servers

Steps to validate the fix

1. Install rke2 with cloud controller manager args
2. Validate if applied
3. Check the logs
4. Validate nodes and pods are up and ok

Validation Results:

$ rke2 -v
rke2 version v1.28.3-rc2+rke2r1 (0d0d0e4879fdf95254461e3a49224f75d7b2dc3d)
go version go1.20.10 X:boringcrypto



$ sudo journalctl -xeu rke2-server.service | grep 'Running cloud'

Oct 25 15:33:00 ip-172-31-2-212 rke2[1778]: time="2023-10-25T15:33:00Z" level=info msg="Running cloud-controller-manager --allocate-node-cidrs=true --authentication-kubeconfig=/var/lib/rancher/rke2/server/cred/cloud-controller.kubeconfig --authorization-kubeconfig=/var/lib/rancher/rke2/server/cred/cloud-controller.kubeconfig --bind-address=127.0.0.1 --cloud-config=/var/lib/rancher/rke2/server/etc/cloud-config.yaml --cloud-provider=rke2 --cluster-cidr=10.42.0.0/24 --configure-cloud-routes=true --controllers=*,-route,-service --feature-gates=CloudDualStackNodeIPs=true --kubeconfig=/var/lib/rancher/rke2/server/cred/cloud-controller.kubeconfig --leader-elect-resource-name=rke2-cloud-controller-manager --node-status-update-frequency=5m0s --profiling=true"


~$ k get nodes 
NAME                                         STATUS   ROLES                       AGE     VERSION
ip-172-31-2-212.us-east-2.compute.internal   Ready    control-plane,etcd,master   8m11s   v1.28.3+rke2r1
ip-172-31-5-191.us-east-2.compute.internal   Ready    control-plane,etcd,master   5m10s   v1.28.3+rke2r1


$ k get pods -A
NAMESPACE     NAME                                                                  READY   STATUS      RESTARTS   AGE
kube-system   cloud-controller-manager-ip-172-31-2-212.us-east-2.compute.internal   1/1     Running     0          4m41s
kube-system   cloud-controller-manager-ip-172-31-5-191.us-east-2.compute.internal   1/1     Running     0          2m40s
kube-system   etcd-ip-172-31-2-212.us-east-2.compute.internal                       1/1     Running     0          5m8s
kube-system   etcd-ip-172-31-5-191.us-east-2.compute.internal                       1/1     Running     0          2m22s
kube-system   helm-install-rke2-canal-pszxh                                         0/1     Completed   0          5m26s
kube-system   helm-install-rke2-coredns-kqwzn                                       0/1     Completed   0          5m26s
kube-system   helm-install-rke2-ingress-nginx-6hhwb                                 0/1     Completed   0          5m26s
kube-system   helm-install-rke2-metrics-server-b5vzc                                0/1     Completed   0          5m26s
kube-system   helm-install-rke2-snapshot-controller-crd-gndhv                       0/1     Completed   0          5m26s
kube-system   helm-install-rke2-snapshot-controller-qtl2z                           0/1     Completed   1          5m26s
kube-system   helm-install-rke2-snapshot-validation-webhook-srg6w                   0/1     Completed   0          5m26s
kube-system   kube-apiserver-ip-172-31-2-212.us-east-2.compute.internal             1/1     Running     0          4m45s
kube-system   kube-apiserver-ip-172-31-5-191.us-east-2.compute.internal             1/1     Running     0          2m42s
kube-system   kube-controller-manager-ip-172-31-2-212.us-east-2.compute.internal    1/1     Running     0          4m47s
kube-system   kube-controller-manager-ip-172-31-5-191.us-east-2.compute.internal    1/1     Running     0          2m40s
kube-system   kube-proxy-ip-172-31-2-212.us-east-2.compute.internal                 1/1     Running     0          4m56s
kube-system   kube-proxy-ip-172-31-5-191.us-east-2.compute.internal                 1/1     Running     0          2m33s
kube-system   kube-scheduler-ip-172-31-2-212.us-east-2.compute.internal             1/1     Running     0          4m50s
kube-system   kube-scheduler-ip-172-31-5-191.us-east-2.compute.internal             1/1     Running     0          2m40s
kube-system   rke2-canal-7qpzz                                                      2/2     Running     0          2m42s
kube-system   rke2-canal-tdd9p                                                      2/2     Running     0          5m3s
kube-system   rke2-coredns-rke2-coredns-6b795db654-nl57d                            1/1     Running     0          5m8s
kube-system   rke2-coredns-rke2-coredns-6b795db654-zpdfj                            1/1     Running     0          2m40s
kube-system   rke2-coredns-rke2-coredns-autoscaler-945fbd459-gw6hr                  1/1     Running     0          5m8s
kube-system   rke2-ingress-nginx-controller-sbmhf                                   1/1     Running     0          3m20s
kube-system   rke2-ingress-nginx-controller-v9r8c                                   0/1     Running     0          94s
kube-system   rke2-metrics-server-544c8c66fc-d7df2                                  1/1     Running     0          3m45s
kube-system   rke2-snapshot-controller-59cc9cd8f4-47ptb                             1/1     Running     0          3m36s
kube-system   rke2-snapshot-validation-webhook-54c5989b65-6hfs6                     1/1     Running     0          3m44s

$ cat /etc/rancher/rke2/config.yaml
write-kubeconfig-mode: "0644"
tls-san:
  - fake.fqdn.value
node-name: ip-172-31-2-212.us-east-2.compute.internal
kube-cloud-controller-manager-arg:
  - allocate-node-cidrs=true
  - authentication-kubeconfig=/var/lib/rancher/rke2/server/cred/cloud-controller.kubeconfig
  - authorization-kubeconfig=/var/lib/rancher/rke2/server/cred/cloud-controller.kubeconfig
  - bind-address=127.0.0.1
  - cloud-config=/var/lib/rancher/rke2/server/etc/cloud-config.yaml
  - cloud-provider=rke2
  - cluster-cidr=10.42.0.0/24
  - configure-cloud-routes=true
  - controllers=*,-route,-service
  - feature-gates=CloudDualStackNodeIPs=true
  - kubeconfig=/var/lib/rancher/rke2/server/cred/cloud-controller.kubeconfig
  - leader-elect-resource-name=rke2-cloud-controller-manager
  - node-status-update-frequency=5m0s
  - profiling=true
node-external-ip: 3.17.76.13