Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

[release-1.25] Validate kube-cloud-controller-manager-arg new arg pulled in from k3s #4950

Closed
rancher-max opened this issue Oct 23, 2023 · 2 comments
Closed

[release-1.25] Validate kube-cloud-controller-manager-arg new arg pulled in from k3s #4950

rancher-max opened this issue Oct 23, 2023 · 2 comments
Assignees
@fmoral2
Milestone
v1.25.15+rke2r1

Comments

@rancher-max
Copy link
Contributor

Backport #4947
@rancher-max rancher-max added this to the v1.25.15+rke2r1 milestone Oct 23, 2023
@fmoral2
Copy link
Contributor

fmoral2 commented Oct 25, 2023

Validated on Version:

-$ rke2 version v1.25.15-rc2+rke2r1 (45f7f1922346518ffab0db448b98a9ebd44b91c2)

Environment Details

Infrastructure
Cloud EC2 instance

Node(s) CPU architecture, OS, and Version:
PRETTY_NAME="Ubuntu 22.04.1 LTS"
NAME="Ubuntu"
VERSION_ID="22.04"

Cluster Configuration:
2 node servers

Steps to validate the fix

  1. Install rke2 with cloud controller manager args
  2. Validate if applied
  3. Check the logs
  4. Validate nodes and pods are up and ok

Validation Results:


$ rke2 -v
rke2 version v1.25.15-rc2+rke2r1 (45f7f1922346518ffab0db448b98a9ebd44b91c2)
go version go1.20.10 X:boringcrypto



$ cat /etc/rancher/rke2/config.yaml
write-kubeconfig-mode: "0644"
tls-san:
  - fake.fqdn.value
node-name: ip-172-31-8-52.us-east-2.compute.internal
kube-cloud-controller-manager-arg:
  - allocate-node-cidrs=true
  - node-status-update-frequency=5m0s



$ sudo journalctl -xeu rke2-server.service | grep 'Running cloud'


Oct 25 16:41:15 ip-172-31-5-165 rke2[49146]: time="2023-10-25T16:41:15Z" level=info msg="Running cloud-controller-manager --allocate-node-cidrs=true --authentication-kubeconfig=/var/lib/rancher/rke2/server/cred/cloud-controller.kubeconfig --authorization-kubeconfig=/var/lib/rancher/rke2/server/cred/cloud-controller.kubeconfig --bind-address=127.0.0.1 --cloud-config=/var/lib/rancher/rke2/server/etc/cloud-config.yaml --cloud-provider=rke2 --cluster-cidr=10.42.0.0/16 --configure-cloud-routes=false --controllers=*,-route,-service --kubeconfig=/var/lib/rancher/rke2/server/cred/cloud-controller.kubeconfig --leader-elect-resource-name=rke2-cloud-controller-manager --node-status-update-frequency=5m0s --profiling=false"

$ k get nodes
NAME                                         STATUS   ROLES                       AGE   VERSION
ip-172-31-8-52.us-east-2.compute.internal    Ready    control-plane,etcd,master   73s   v1.25.15+rke2r1
ip-172-31-9-177.us-east-2.compute.internal   Ready    control-plane,etcd,master   37m   v1.25.15+rke2r1


$ k get pods -A
NAMESPACE     NAME                                                                  READY   STATUS             RESTARTS         AGE
kube-system   cloud-controller-manager-ip-172-31-8-52.us-east-2.compute.internal    1/1     Running            1 (93s ago)      104s
kube-system   etcd-ip-172-31-5-165.us-east-2.compute.internal                       1/1     Running            0                40m
kube-system   etcd-ip-172-31-8-52.us-east-2.compute.internal                        1/1     Running            0                103s
kube-system   etcd-ip-172-31-9-177.us-east-2.compute.internal                       1/1     Running            0                37m
kube-system   helm-install-rke2-canal-h85kq                                         0/1     Completed          0                40m
kube-system   helm-install-rke2-coredns-k87rm                                       0/1     Completed          0                40m
kube-system   helm-install-rke2-ingress-nginx-l8fpl                                 0/1     Pending            0                40m
kube-system   helm-install-rke2-metrics-server-ld2tc                                0/1     Pending            0                40m
kube-system   helm-install-rke2-snapshot-controller-crd-lslsw                       0/1     Pending            0                40m
kube-system   helm-install-rke2-snapshot-controller-tl4q2                           0/1     Pending            0                40m
kube-system   helm-install-rke2-snapshot-validation-webhook-788sc                   0/1     Pending            0                40m
kube-system   kube-apiserver-ip-172-31-5-165.us-east-2.compute.internal             1/1     Running            0                40m
kube-system   kube-apiserver-ip-172-31-8-52.us-east-2.compute.internal              1/1     Running            0                103s
kube-system   kube-apiserver-ip-172-31-9-177.us-east-2.compute.internal             1/1     Running            0                37m
kube-system   kube-controller-manager-ip-172-31-5-165.us-east-2.compute.internal    1/1     Running            0                40m
kube-system   kube-controller-manager-ip-172-31-8-52.us-east-2.compute.internal     1/1     Running            1 (91s ago)      104s
kube-system   kube-controller-manager-ip-172-31-9-177.us-east-2.compute.internal    1/1     Running            0                37m
kube-system   kube-proxy-ip-172-31-5-165.us-east-2.compute.internal                 1/1     Running            0                40m
kube-system   kube-proxy-ip-172-31-8-52.us-east-2.compute.internal                  1/1     Running            0                31s
kube-system   kube-proxy-ip-172-31-9-177.us-east-2.compute.internal                 1/1     Running            0                37m
kube-system   kube-scheduler-ip-172-31-5-165.us-east-2.compute.internal             1/1     Running            0                40m
kube-system   kube-scheduler-ip-172-31-8-52.us-east-2.compute.internal              1/1     Running            0                104s
kube-system   kube-scheduler-ip-172-31-9-177.us-east-2.compute.internal             1/1     Running            0                37m
kube-system   rke2-canal-9z44j                                                      2/2     Running            0                37m
kube-system   rke2-canal-trn6z                                                      2/2     Running            0                105s
kube-system   rke2-canal-xvprk                                                      2/2     Running            0                39m
kube-system   rke2-coredns-rke2-coredns-657cc8fbcc-msdsq                            0/1     Pending            0                39m
kube-system   rke2-coredns-rke2-coredns-autoscaler-6cc6bfffcf-bjc2x                 0/1     Pending            0                39m

@fmoral2 fmoral2 closed this as completed Oct 25, 2023
@fmoral2
Copy link
Contributor

fmoral2 commented Oct 26, 2023

Validated on Version:

-$ rke2 version v1.25.15-rc2+rke2r1 (45f7f1922346518ffab0db448b98a9ebd44b91c2)

Environment Details

Infrastructure
Cloud EC2 instance

Node(s) CPU architecture, OS, and Version:
PRETTY_NAME="Ubuntu 22.04.1 LTS"
NAME="Ubuntu"
VERSION_ID="22.04"

Cluster Configuration:
2 node servers

Steps to validate the fix

  1. Install rke2 with cloud controller manager args
  2. Validate if applied
  3. Check the logs
  4. Validate nodes and pods are up and ok

Validation Results:


$ rke2 -v
rke2 version v1.25.15-rc2+rke2r1 (45f7f1922346518ffab0db448b98a9ebd44b91c2)
go version go1.20.10 X:boringcrypto



$ cat /etc/rancher/rke2/config.yaml
write-kubeconfig-mode: "0644"
tls-san:
  - fake.fqdn.value
node-name: ip-172-31-8-52.us-east-2.compute.internal
kube-cloud-controller-manager-arg:
  - allocate-node-cidrs=true
  - node-status-update-frequency=5m0s



$ sudo journalctl -xeu rke2-server.service | grep 'Running cloud'


Oct 25 16:41:15 ip-172-31-5-165 rke2[49146]: time="2023-10-25T16:41:15Z" level=info msg="Running cloud-controller-manager --allocate-node-cidrs=true --authentication-kubeconfig=/var/lib/rancher/rke2/server/cred/cloud-controller.kubeconfig --authorization-kubeconfig=/var/lib/rancher/rke2/server/cred/cloud-controller.kubeconfig --bind-address=127.0.0.1 --cloud-config=/var/lib/rancher/rke2/server/etc/cloud-config.yaml --cloud-provider=rke2 --cluster-cidr=10.42.0.0/16 --configure-cloud-routes=false --controllers=*,-route,-service --kubeconfig=/var/lib/rancher/rke2/server/cred/cloud-controller.kubeconfig --leader-elect-resource-name=rke2-cloud-controller-manager --node-status-update-frequency=5m0s --profiling=false"

$ k get nodes
NAME                                         STATUS   ROLES                       AGE   VERSION
ip-172-31-8-52.us-east-2.compute.internal    Ready    control-plane,etcd,master   73s   v1.25.15+rke2r1
ip-172-31-9-177.us-east-2.compute.internal   Ready    control-plane,etcd,master   37m   v1.25.15+rke2r1


$ k get pods -A
NAMESPACE     NAME                                                                  READY   STATUS             RESTARTS         AGE
kube-system   cloud-controller-manager-ip-172-31-8-52.us-east-2.compute.internal    1/1     Running            1 (93s ago)      104s
kube-system   etcd-ip-172-31-5-165.us-east-2.compute.internal                       1/1     Running            0                40m
kube-system   etcd-ip-172-31-8-52.us-east-2.compute.internal                        1/1     Running            0                103s
kube-system   etcd-ip-172-31-9-177.us-east-2.compute.internal                       1/1     Running            0                37m
kube-system   helm-install-rke2-canal-h85kq                                         0/1     Completed          0                40m
kube-system   helm-install-rke2-coredns-k87rm                                       0/1     Completed          0                40m
kube-system   helm-install-rke2-ingress-nginx-l8fpl                                 0/1     Pending            0                40m
kube-system   helm-install-rke2-metrics-server-ld2tc                                0/1     Pending            0                40m
kube-system   helm-install-rke2-snapshot-controller-crd-lslsw                       0/1     Pending            0                40m
kube-system   helm-install-rke2-snapshot-controller-tl4q2                           0/1     Pending            0                40m
kube-system   helm-install-rke2-snapshot-validation-webhook-788sc                   0/1     Pending            0                40m
kube-system   kube-apiserver-ip-172-31-5-165.us-east-2.compute.internal             1/1     Running            0                40m
kube-system   kube-apiserver-ip-172-31-8-52.us-east-2.compute.internal              1/1     Running            0                103s
kube-system   kube-apiserver-ip-172-31-9-177.us-east-2.compute.internal             1/1     Running            0                37m
kube-system   kube-controller-manager-ip-172-31-5-165.us-east-2.compute.internal    1/1     Running            0                40m
kube-system   kube-controller-manager-ip-172-31-8-52.us-east-2.compute.internal     1/1     Running            1 (91s ago)      104s
kube-system   kube-controller-manager-ip-172-31-9-177.us-east-2.compute.internal    1/1     Running            0                37m
kube-system   kube-proxy-ip-172-31-5-165.us-east-2.compute.internal                 1/1     Running            0                40m
kube-system   kube-proxy-ip-172-31-8-52.us-east-2.compute.internal                  1/1     Running            0                31s
kube-system   kube-proxy-ip-172-31-9-177.us-east-2.compute.internal                 1/1     Running            0                37m
kube-system   kube-scheduler-ip-172-31-5-165.us-east-2.compute.internal             1/1     Running            0                40m
kube-system   kube-scheduler-ip-172-31-8-52.us-east-2.compute.internal              1/1     Running            0                104s
kube-system   kube-scheduler-ip-172-31-9-177.us-east-2.compute.internal             1/1     Running            0                37m
kube-system   rke2-canal-9z44j                                                      2/2     Running            0                37m
kube-system   rke2-canal-trn6z                                                      2/2     Running            0                105s
kube-system   rke2-canal-xvprk                                                      2/2     Running            0                39m
kube-system   rke2-coredns-rke2-coredns-657cc8fbcc-msdsq                            0/1     Pending            0                39m
kube-system   rke2-coredns-rke2-coredns-autoscaler-6cc6bfffcf-bjc2x                 0/1     Pending            0                39m

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees

@fmoral2 fmoral2

Labels
None yet
Projects
None yet
Milestone
v1.25.15+rke2r1
Development

No branches or pull requests
2 participants
@rancher-max @fmoral2