Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

[release-1.26] Validate kube-cloud-controller-manager-arg new arg pulled in from k3s #4949

Closed
rancher-max opened this issue Oct 23, 2023 · 2 comments
Closed

[release-1.26] Validate kube-cloud-controller-manager-arg new arg pulled in from k3s #4949

rancher-max opened this issue Oct 23, 2023 · 2 comments
Assignees
@fmoral2
Milestone
v1.26.10+rke2r1

Comments

@rancher-max
Copy link
Contributor

Backport #4947
@rancher-max rancher-max added this to the v1.26.10+rke2r1 milestone Oct 23, 2023
@fmoral2
Copy link
Contributor

fmoral2 commented Oct 25, 2023

Validated on Version:

-$ rke2 version v1.26.10-rc2+rke2r1 (825e3188d273e7271a0b5ce924d42455b4d37a34)

Environment Details

Infrastructure
Cloud EC2 instance

Node(s) CPU architecture, OS, and Version:
PRETTY_NAME="Ubuntu 22.04.1 LTS"
NAME="Ubuntu"
VERSION_ID="22.04"

Cluster Configuration:
2 node servers

Steps to validate the fix

  1. Install rke2 with cloud controller manager args
  2. Validate if applied
  3. Check the logs
  4. Validate nodes and pods are up and ok

Validation Results:


$ rke2 -v
rke2 version v1.26.10-rc2+rke2r1 (825e3188d273e7271a0b5ce924d42455b4d37a34)
go version go1.20.10 X:boringcrypto

$ cat /etc/rancher/rke2/config.yaml
write-kubeconfig-mode: "0644"
tls-san:
  - fake.fqdn.value
node-name: ip-172-31-8-52.us-east-2.compute.internal
kube-cloud-controller-manager-arg:
  - allocate-node-cidrs=true
  - node-status-update-frequency=5m0s



$ sudo journalctl -xeu rke2-server.service | grep 'Running cloud'

Oct 25 16:37:57 ip-172-31-12-248 rke2[60701]: time="2023-10-25T16:37:57Z" level=info msg="Running cloud-controller-manager --allocate-node-cidrs=true --authentication-kubeconfig=/var/lib/rancher/rke2/server/cred/cloud-controller.kubeconfig --authorization-kubeconfig=/var/lib/rancher/rke2/server/cred/cloud-controller.kubeconfig --bind-address=127.0.0.1 --cloud-config=/var/lib/rancher/rke2/server/etc/cloud-config.yaml --cloud-provider=rke2 --cluster-cidr=10.42.0.0/16 --configure-cloud-routes=false --controllers=*,-route,-service --kubeconfig=/var/lib/rancher/rke2/server/cred/cloud-controller.kubeconfig --leader-elect-resource-name=rke2-cloud-controller-manager --node-status-update-frequency=5m0s --profiling=false"



$ k get nodes
NAME                                          STATUS     ROLES                       AGE     VERSION
ip-172-31-11-99.us-east-2.compute.internal    Ready      control-plane,etcd,master   43m     v1.26.10+rke2r1
ip-172-31-8-52.us-east-2.compute.internal     Ready      control-plane,etcd,master   5m49s   v1.26.10+rke2r1



$ k get pods -A
NAMESPACE     NAME                                                                   READY   STATUS             RESTARTS         AGE
kube-system   cloud-controller-manager-ip-172-31-8-52.us-east-2.compute.internal     1/1     Running            0                2m13s
kube-system   etcd-ip-172-31-11-99.us-east-2.compute.internal                        1/1     Running            0                42m
kube-system   etcd-ip-172-31-12-248.us-east-2.compute.internal                       1/1     Running            0                49m
kube-system   etcd-ip-172-31-8-52.us-east-2.compute.internal                         1/1     Running            0                6m4s
kube-system   helm-install-rke2-canal-zgpjq                                          0/1     Completed          0                49m
kube-system   helm-install-rke2-coredns-jfnpm                                        0/1     Completed          0                49m
kube-system   helm-install-rke2-ingress-nginx-94kbc                                  0/1     Completed          0                49m
kube-system   helm-install-rke2-metrics-server-44fcm                                 0/1     Completed          0                49m
kube-system   helm-install-rke2-snapshot-controller-72265                            0/1     Completed          1                49m
kube-system   helm-install-rke2-snapshot-controller-crd-nbdhs                        0/1     Completed          0                49m
kube-system   helm-install-rke2-snapshot-validation-webhook-w7mdj                    0/1     Completed          0                49m
kube-system   kube-apiserver-ip-172-31-11-99.us-east-2.compute.internal              1/1     Running            0                42m
kube-system   kube-apiserver-ip-172-31-12-248.us-east-2.compute.internal             1/1     Running            0                48m
kube-system   kube-apiserver-ip-172-31-8-52.us-east-2.compute.internal               1/1     Running            0                6m4s
kube-system   kube-controller-manager-ip-172-31-11-99.us-east-2.compute.internal     1/1     Running            0                42m
kube-system   kube-controller-manager-ip-172-31-12-248.us-east-2.compute.internal    1/1     Running            0                49m
kube-system   kube-controller-manager-ip-172-31-8-52.us-east-2.compute.internal      1/1     Running            1 (5m52s ago)    6m5s
kube-system   kube-proxy-ip-172-31-11-99.us-east-2.compute.internal                  1/1     Running            0                43m
kube-system   kube-proxy-ip-172-31-12-248.us-east-2.compute.internal                 1/1     Running            0                49m
kube-system   kube-proxy-ip-172-31-8-52.us-east-2.compute.internal                   1/1     Running            0                4m55s
kube-system   kube-scheduler-ip-172-31-11-99.us-east-2.compute.internal              1/1     Running            0                42m
kube-system   kube-scheduler-ip-172-31-12-248.us-east-2.compute.internal             1/1     Running            0                49m
kube-system   kube-scheduler-ip-172-31-8-52.us-east-2.compute.internal               1/1     Running            1 (5m52s ago)    6m4s
kube-system   rke2-canal-2fppr                                                       2/2     Running            0                43m
kube-system   rke2-canal-s7wzk                                                       2/2     Running            0                48m
kube-system   rke2-canal-zp6st                                                       2/2     Running            0                6m5s
kube-system   rke2-coredns-rke2-coredns-565dfc7d75-cr7hh                             1/1     Running            0                101s
kube-system   rke2-coredns-rke2-coredns-565dfc7d75-ffv6w                             1/1     Running            0                48m
kube-system   rke2-coredns-rke2-coredns-autoscaler-6c48c95bf9-f6xkh                  1/1     Running            0                48m
kube-system   rke2-ingress-nginx-controller-cpwds                                    0/1     Running            0                68s
kube-system   rke2-ingress-nginx-controller-srg2w                                    1/1     Running            0                68s
kube-system   rke2-metrics-server-c9c78bd66-hcl7l                                    1/1     Running            0                84s
kube-system   rke2-snapshot-controller-6f7bbb497d-gq4h7                              1/1     Running            0                77s
kube-system   rke2-snapshot-validation-webhook-65b5675d5c-9zvrq                      1/1     Running            0                84s

@fmoral2 fmoral2 closed this as completed Oct 25, 2023
@fmoral2
Copy link
Contributor

fmoral2 commented Oct 26, 2023

Validated on Version:

-$ rke2 version v1.26.10-rc2+rke2r1 (825e3188d273e7271a0b5ce924d42455b4d37a34)

Environment Details

Infrastructure
Cloud EC2 instance

Node(s) CPU architecture, OS, and Version:
PRETTY_NAME="Ubuntu 22.04.1 LTS"
NAME="Ubuntu"
VERSION_ID="22.04"

Cluster Configuration:
2 node servers

Steps to validate the fix

  1. Install rke2 with cloud controller manager args
  2. Validate if applied
  3. Check the logs
  4. Validate nodes and pods are up and ok

Validation Results:


$ rke2 -v
rke2 version v1.26.10-rc2+rke2r1 (825e3188d273e7271a0b5ce924d42455b4d37a34)
go version go1.20.10 X:boringcrypto

$ cat /etc/rancher/rke2/config.yaml
write-kubeconfig-mode: "0644"
tls-san:
  - fake.fqdn.value
node-name: ip-172-31-8-52.us-east-2.compute.internal
kube-cloud-controller-manager-arg:
  - allocate-node-cidrs=true
  - node-status-update-frequency=5m0s



$ sudo journalctl -xeu rke2-server.service | grep 'Running cloud'

Oct 25 16:37:57 ip-172-31-12-248 rke2[60701]: time="2023-10-25T16:37:57Z" level=info msg="Running cloud-controller-manager --allocate-node-cidrs=true --authentication-kubeconfig=/var/lib/rancher/rke2/server/cred/cloud-controller.kubeconfig --authorization-kubeconfig=/var/lib/rancher/rke2/server/cred/cloud-controller.kubeconfig --bind-address=127.0.0.1 --cloud-config=/var/lib/rancher/rke2/server/etc/cloud-config.yaml --cloud-provider=rke2 --cluster-cidr=10.42.0.0/16 --configure-cloud-routes=false --controllers=*,-route,-service --kubeconfig=/var/lib/rancher/rke2/server/cred/cloud-controller.kubeconfig --leader-elect-resource-name=rke2-cloud-controller-manager --node-status-update-frequency=5m0s --profiling=false"



$ k get nodes
NAME                                          STATUS     ROLES                       AGE     VERSION
ip-172-31-11-99.us-east-2.compute.internal    Ready      control-plane,etcd,master   43m     v1.26.10+rke2r1
ip-172-31-8-52.us-east-2.compute.internal     Ready      control-plane,etcd,master   5m49s   v1.26.10+rke2r1



$ k get pods -A
NAMESPACE     NAME                                                                   READY   STATUS             RESTARTS         AGE
kube-system   cloud-controller-manager-ip-172-31-8-52.us-east-2.compute.internal     1/1     Running            0                2m13s
kube-system   etcd-ip-172-31-11-99.us-east-2.compute.internal                        1/1     Running            0                42m
kube-system   etcd-ip-172-31-12-248.us-east-2.compute.internal                       1/1     Running            0                49m
kube-system   etcd-ip-172-31-8-52.us-east-2.compute.internal                         1/1     Running            0                6m4s
kube-system   helm-install-rke2-canal-zgpjq                                          0/1     Completed          0                49m
kube-system   helm-install-rke2-coredns-jfnpm                                        0/1     Completed          0                49m
kube-system   helm-install-rke2-ingress-nginx-94kbc                                  0/1     Completed          0                49m
kube-system   helm-install-rke2-metrics-server-44fcm                                 0/1     Completed          0                49m
kube-system   helm-install-rke2-snapshot-controller-72265                            0/1     Completed          1                49m
kube-system   helm-install-rke2-snapshot-controller-crd-nbdhs                        0/1     Completed          0                49m
kube-system   helm-install-rke2-snapshot-validation-webhook-w7mdj                    0/1     Completed          0                49m
kube-system   kube-apiserver-ip-172-31-11-99.us-east-2.compute.internal              1/1     Running            0                42m
kube-system   kube-apiserver-ip-172-31-12-248.us-east-2.compute.internal             1/1     Running            0                48m
kube-system   kube-apiserver-ip-172-31-8-52.us-east-2.compute.internal               1/1     Running            0                6m4s
kube-system   kube-controller-manager-ip-172-31-11-99.us-east-2.compute.internal     1/1     Running            0                42m
kube-system   kube-controller-manager-ip-172-31-12-248.us-east-2.compute.internal    1/1     Running            0                49m
kube-system   kube-controller-manager-ip-172-31-8-52.us-east-2.compute.internal      1/1     Running            1 (5m52s ago)    6m5s
kube-system   kube-proxy-ip-172-31-11-99.us-east-2.compute.internal                  1/1     Running            0                43m
kube-system   kube-proxy-ip-172-31-12-248.us-east-2.compute.internal                 1/1     Running            0                49m
kube-system   kube-proxy-ip-172-31-8-52.us-east-2.compute.internal                   1/1     Running            0                4m55s
kube-system   kube-scheduler-ip-172-31-11-99.us-east-2.compute.internal              1/1     Running            0                42m
kube-system   kube-scheduler-ip-172-31-12-248.us-east-2.compute.internal             1/1     Running            0                49m
kube-system   kube-scheduler-ip-172-31-8-52.us-east-2.compute.internal               1/1     Running            1 (5m52s ago)    6m4s
kube-system   rke2-canal-2fppr                                                       2/2     Running            0                43m
kube-system   rke2-canal-s7wzk                                                       2/2     Running            0                48m
kube-system   rke2-canal-zp6st                                                       2/2     Running            0                6m5s
kube-system   rke2-coredns-rke2-coredns-565dfc7d75-cr7hh                             1/1     Running            0                101s
kube-system   rke2-coredns-rke2-coredns-565dfc7d75-ffv6w                             1/1     Running            0                48m
kube-system   rke2-coredns-rke2-coredns-autoscaler-6c48c95bf9-f6xkh                  1/1     Running            0                48m
kube-system   rke2-ingress-nginx-controller-cpwds                                    0/1     Running            0                68s
kube-system   rke2-ingress-nginx-controller-srg2w                                    1/1     Running            0                68s
kube-system   rke2-metrics-server-c9c78bd66-hcl7l                                    1/1     Running            0                84s
kube-system   rke2-snapshot-controller-6f7bbb497d-gq4h7                              1/1     Running            0                77s
kube-system   rke2-snapshot-validation-webhook-65b5675d5c-9zvrq                      1/1     Running            0                84s

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees

@fmoral2 fmoral2

Labels
None yet
Projects
None yet
Milestone
v1.26.10+rke2r1
Development

No branches or pull requests
2 participants
@rancher-max @fmoral2