Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

[release-1.28] DaemonSet for ContainerNetworking DHCP CNI Plugin #5207

Closed
thomasferrandiz opened this issue Jan 5, 2024 · 1 comment
Closed

[release-1.28] DaemonSet for ContainerNetworking DHCP CNI Plugin #5207

thomasferrandiz opened this issue Jan 5, 2024 · 1 comment
Assignees
@endawkins
Milestone
v1.28.6+rke2r1

Comments

@thomasferrandiz
Copy link
Contributor

Backport of: #3917
@thomasferrandiz thomasferrandiz added this to the v1.28.6+rke2r1 milestone Jan 5, 2024
@thomasferrandiz thomasferrandiz mentioned this issue Jan 5, 2024
Merged
@endawkins
Copy link

Validated on 1.28 with 26d0a97 / version 1.28

Environment Details

Infrastructure

  • Cloud
  • Hosted

Node(s) CPU architecture, OS, and Version:

Linux ip-172-31-30-106 6.2.0-1017-aws #17~22.04.1-Ubuntu SMP Fri Nov 17 21:07:13 UTC 2023 x86_64 x86_64 x86_64 GNU/Linux
PRETTY_NAME="Ubuntu 22.04.3 LTS"
NAME="Ubuntu"
VERSION_ID="22.04"
VERSION="22.04.3 LTS (Jammy Jellyfish)"
VERSION_CODENAME=jammy
ID=ubuntu
ID_LIKE=debian
HOME_URL="https://www.ubuntu.com/"
SUPPORT_URL="https://help.ubuntu.com/"
BUG_REPORT_URL="https://bugs.launchpad.net/ubuntu/"
PRIVACY_POLICY_URL="https://www.ubuntu.com/legal/terms-and-policies/privacy-policy"
UBUNTU_CODENAME=jammy

Cluster Configuration:

1 server
1 agent

Config.yaml:

write-kubeconfig-mode: 644
token: test
node-external-ip: <EXTERNAL_IP>
cni: multus,calico

Additional files

***rke2-multus-config.yaml

apiVersion: helm.cattle.io/v1
kind: HelmChartConfig
metadata:
  name: rke2-multus
  namespace: kube-system
spec:
  valuesContent: |-
    manifests:
      dhcpDaemonSet: true

Testing Steps

  1. Copy config.yaml
$ sudo mkdir -p /etc/rancher/rke2 && sudo cp config.yaml /etc/rancher/rke2
  1. Install RKE2 (Do not enable or start rke2)
  2. Create file /var/lib/rancher/rke2/server/manifests/rke2-multus-config.yaml
  3. Enable and start rke2 (both server and agent)
  4. Verify that kube-rke2-multus-dhcp was created and started properly
  5. Verify /run/cni/dhcp.sock was created on agent node

Replication Results:

  • rke2 version used for replication:
$ rke2 -v
rke2 version v1.28.5+rke2r1 (adcd936ed6d7280080fba3670208cdea532de453)
go version go1.20.12 X:boringcrypto

$ kubectl get nodes,pods -A -o wide                                                                                                                                                        ip-172-31-17-51: Wed Jan 24 16:41:31 2024

NAME                    STATUS   ROLES                       AGE     VERSION          INTERNAL-IP     EXTERNAL-IP     OS-IMAGE             KERNEL-VERSION   CONTAINER-RUNTIME
node/ip-172-31-17-51    Ready    control-plane,etcd,master   3m33s   v1.28.5+rke2r1   172.31.17.51    <EXTERNAL_IP>   Ubuntu 22.04.3 LTS   6.2.0-1017-aws   containerd://1.7.11-k3s2
node/ip-172-31-18-234   Ready    <none>                      110s    v1.28.5+rke2r1   172.31.18.234   <none>          Ubuntu 22.04.3 LTS   6.2.0-1017-aws   containerd://1.7.11-k3s2

NAMESPACE         NAME                                                       READY   STATUS      RESTARTS        AGE     IP              NODE               NOMINATED NODE   READINESS GATES
calico-system     pod/calico-kube-controllers-8c87fb85b-7tw8n                1/1     Running     0               2m36s   10.42.72.69     ip-172-31-17-51    <none>           <none>
calico-system     pod/calico-node-7ztr9                                      1/1     Running     0               2m36s   172.31.17.51    ip-172-31-17-51    <none>           <none>
calico-system     pod/calico-node-9wsdd                                      1/1     Running     0               110s    172.31.18.234   ip-172-31-18-234   <none>           <none>
calico-system     pod/calico-typha-765874896d-nddlc                          1/1     Running     0               2m36s   172.31.17.51    ip-172-31-17-51    <none>           <none>
kube-system       pod/cloud-controller-manager-ip-172-31-17-51               1/1     Running     0               3m30s   172.31.17.51    ip-172-31-17-51    <none>           <none>
kube-system       pod/etcd-ip-172-31-17-51                                   1/1     Running     0               3m      172.31.17.51    ip-172-31-17-51    <none>           <none>
kube-system       pod/helm-install-rke2-calico-crd-9tz78                     0/1     Completed   0               3m16s   172.31.17.51    ip-172-31-17-51    <none>           <none>
kube-system       pod/helm-install-rke2-calico-w259q                         0/1     Completed   2               3m16s   172.31.17.51    ip-172-31-17-51    <none>           <none>
kube-system       pod/helm-install-rke2-coredns-5zf68                        0/1     Completed   0               3m16s   172.31.17.51    ip-172-31-17-51    <none>           <none>
kube-system       pod/helm-install-rke2-ingress-nginx-6slxs                  0/1     Completed   0               3m16s   10.42.72.68     ip-172-31-17-51    <none>           <none>
kube-system       pod/helm-install-rke2-metrics-server-7vrj5                 0/1     Completed   0               3m16s   10.42.72.65     ip-172-31-17-51    <none>           <none>
kube-system       pod/helm-install-rke2-multus-52n6f                         0/1     Completed   0               3m15s   172.31.17.51    ip-172-31-17-51    <none>           <none>
kube-system       pod/helm-install-rke2-snapshot-controller-crd-t95lp        0/1     Completed   0               3m15s   10.42.72.72     ip-172-31-17-51    <none>           <none>
kube-system       pod/helm-install-rke2-snapshot-controller-dv7f4            0/1     Completed   1               3m15s   10.42.72.67     ip-172-31-17-51    <none>           <none>
kube-system       pod/helm-install-rke2-snapshot-validation-webhook-mgfw8    0/1     Completed   0               3m15s   10.42.72.71     ip-172-31-17-51    <none>           <none>
kube-system       pod/kube-apiserver-ip-172-31-17-51                         1/1     Running     0               3m33s   172.31.17.51    ip-172-31-17-51    <none>           <none>
kube-system       pod/kube-controller-manager-ip-172-31-17-51                1/1     Running     0               3m32s   172.31.17.51    ip-172-31-17-51    <none>           <none>
kube-system       pod/kube-proxy-ip-172-31-17-51                             1/1     Running     0               3m25s   172.31.17.51    ip-172-31-17-51    <none>           <none>
kube-system       pod/kube-proxy-ip-172-31-18-234                            1/1     Running     0               110s    172.31.18.234   ip-172-31-18-234   <none>           <none>
kube-system       pod/kube-scheduler-ip-172-31-17-51                         1/1     Running     0               3m32s   172.31.17.51    ip-172-31-17-51    <none>           <none>
kube-system       pod/rke2-coredns-rke2-coredns-6b795db654-78ssq             1/1     Running     0               3m5s    10.42.72.70     ip-172-31-17-51    <none>           <none>
kube-system       pod/rke2-coredns-rke2-coredns-6b795db654-zfw9b             1/1     Running     0               102s    10.42.2.192     ip-172-31-18-234   <none>           <none>
kube-system       pod/rke2-coredns-rke2-coredns-autoscaler-945fbd459-ls8xb   1/1     Running     0               3m5s    10.42.72.66     ip-172-31-17-51    <none>           <none>
kube-system       pod/rke2-ingress-nginx-controller-6jks2                    1/1     Running     0               56s     10.42.2.193     ip-172-31-18-234   <none>           <none>
kube-system       pod/rke2-ingress-nginx-controller-n9b5j                    1/1     Running     0               94s     10.42.72.77     ip-172-31-17-51    <none>           <none>
kube-system       pod/rke2-metrics-server-544c8c66fc-x8t9q                   1/1     Running     0               115s    10.42.72.73     ip-172-31-17-51    <none>           <none>
kube-system       pod/rke2-multus-ds-jqgwv                                   1/1     Running     3 (67s ago)     110s    172.31.18.234   ip-172-31-18-234   <none>           <none>
kube-system       pod/rke2-multus-ds-qc7vb                                   1/1     Running     3 (2m31s ago)   3m7s    172.31.17.51    ip-172-31-17-51    <none>           <none>
kube-system       pod/rke2-snapshot-controller-59cc9cd8f4-2k6s2              1/1     Running     0               105s    10.42.72.74     ip-172-31-17-51    <none>           <none>
kube-system       pod/rke2-snapshot-validation-webhook-54c5989b65-9fmn5      1/1     Running     0               104s    10.42.72.76     ip-172-31-17-51    <none>           <none>
tigera-operator   pod/tigera-operator-59d6c9b46-jvxmx                        1/1     Running     0               2m44s   172.31.17.51    ip-172-31-17-51    <none>           <none>

$ ls -l /run/cni/
ls: cannot access '/run/cni/': No such file or directory

Validation Results:

  • rke2 version used for validation:
$ rke2 -v
rke2 version v1.28.6-rc2+rke2r1 (26d0a97e92fcf5e2786784b30d96096bbeb9d537)
go version go1.20.13 X:boringcrypto

$ kubectl get nodes,pods -A -o wide                                                                                                                                                       ip-172-31-30-106: Wed Jan 24 17:48:32 2024

NAME                    STATUS   ROLES                       AGE     VERSION          INTERNAL-IP     EXTERNAL-IP     OS-IMAGE             KERNEL-VERSION   CONTAINER-RUNTIME
node/ip-172-31-24-197   Ready    <none>                      116s    v1.28.6+rke2r1   172.31.24.197   <none>          Ubuntu 22.04.3 LTS   6.2.0-1017-aws   containerd://1.7.11-k3s2
node/ip-172-31-30-106   Ready    control-plane,etcd,master   4m25s   v1.28.6+rke2r1   172.31.30.106   <EXTERNAL_IP>   Ubuntu 22.04.3 LTS   6.2.0-1017-aws   containerd://1.7.11-k3s2

NAMESPACE         NAME                                                        READY   STATUS      RESTARTS        AGE     IP              NODE               NOMINATED NODE   READINESS GATES
calico-system     pod/calico-kube-controllers-5d98fddbc5-qrbkj                1/1     Running     0               3m23s   10.42.17.135    ip-172-31-30-106   <none>           <none>
calico-system     pod/calico-node-gmshx                                       1/1     Running     0               3m24s   172.31.30.106   ip-172-31-30-106   <none>           <none>
calico-system     pod/calico-node-tslpk                                       1/1     Running     0               116s    172.31.24.197   ip-172-31-24-197   <none>           <none>
calico-system     pod/calico-typha-7b5f487444-cc9d4                           1/1     Running     0               3m24s   172.31.30.106   ip-172-31-30-106   <none>           <none>
kube-system       pod/cloud-controller-manager-ip-172-31-30-106               1/1     Running     0               4m1s    172.31.30.106   ip-172-31-30-106   <none>           <none>
kube-system       pod/etcd-ip-172-31-30-106                                   1/1     Running     0               3m30s   172.31.30.106   ip-172-31-30-106   <none>           <none>
kube-system       pod/helm-install-rke2-calico-crd-s6qg9                      0/1     Completed   0               4m7s    172.31.30.106   ip-172-31-30-106   <none>           <none>
kube-system       pod/helm-install-rke2-calico-rnvxm                          0/1     Completed   2               4m7s    172.31.30.106   ip-172-31-30-106   <none>           <none>
kube-system       pod/helm-install-rke2-coredns-gk8nb                         0/1     Completed   0               4m7s    172.31.30.106   ip-172-31-30-106   <none>           <none>
kube-system       pod/helm-install-rke2-ingress-nginx-vtgvb                   0/1     Completed   0               4m7s    10.42.17.130    ip-172-31-30-106   <none>           <none>
kube-system       pod/helm-install-rke2-metrics-server-4zgl9                  0/1     Completed   0               4m7s    10.42.17.129    ip-172-31-30-106   <none>           <none>
kube-system       pod/helm-install-rke2-multus-z78zb                          0/1     Completed   0               4m7s    172.31.30.106   ip-172-31-30-106   <none>           <none>
kube-system       pod/helm-install-rke2-snapshot-controller-crd-w7vsj         0/1     Completed   0               4m7s    10.42.17.133    ip-172-31-30-106   <none>           <none>
kube-system       pod/helm-install-rke2-snapshot-controller-nv4z6             0/1     Completed   1               4m7s    10.42.17.131    ip-172-31-30-106   <none>           <none>
kube-system       pod/helm-install-rke2-snapshot-validation-webhook-rt5vt     0/1     Completed   0               4m7s    10.42.17.132    ip-172-31-30-106   <none>           <none>
kube-system       pod/kube-apiserver-ip-172-31-30-106                         1/1     Running     0               3m37s   172.31.30.106   ip-172-31-30-106   <none>           <none>
kube-system       pod/kube-controller-manager-ip-172-31-30-106                1/1     Running     0               3m23s   172.31.30.106   ip-172-31-30-106   <none>           <none>
kube-system       pod/kube-proxy-ip-172-31-24-197                             1/1     Running     0               115s    172.31.24.197   ip-172-31-24-197   <none>           <none>
kube-system       pod/kube-proxy-ip-172-31-30-106                             1/1     Running     0               3m59s   172.31.30.106   ip-172-31-30-106   <none>           <none>
kube-system       pod/kube-scheduler-ip-172-31-30-106                         1/1     Running     0               3m22s   172.31.30.106   ip-172-31-30-106   <none>           <none>
kube-system       pod/rke2-coredns-rke2-coredns-78f5dbb485-47mz2              1/1     Running     0               110s    10.42.243.128   ip-172-31-24-197   <none>           <none>
kube-system       pod/rke2-coredns-rke2-coredns-78f5dbb485-rwjj4              1/1     Running     0               3m50s   10.42.17.134    ip-172-31-30-106   <none>           <none>
kube-system       pod/rke2-coredns-rke2-coredns-autoscaler-64b867c686-64sqz   1/1     Running     0               3m50s   10.42.17.136    ip-172-31-30-106   <none>           <none>
kube-system       pod/rke2-ingress-nginx-controller-2ddxr                     1/1     Running     0               61s     10.42.243.129   ip-172-31-24-197   <none>           <none>
kube-system       pod/rke2-ingress-nginx-controller-47kl7                     1/1     Running     0               109s    10.42.17.141    ip-172-31-30-106   <none>           <none>
kube-system       pod/rke2-metrics-server-544c8c66fc-qwp9t                    1/1     Running     0               2m7s    10.42.17.137    ip-172-31-30-106   <none>           <none>
kube-system       pod/rke2-multus-cz5lq                                       1/1     Running     3 (3m12s ago)   3m51s   172.31.30.106   ip-172-31-30-106   <none>           <none>
kube-system       pod/rke2-multus-dhcp-jhhsk                                  1/1     Running     0               61s     172.31.24.197   ip-172-31-24-197   <none>           <none>
kube-system       pod/rke2-multus-dhcp-kcdd9                                  1/1     Running     0               2m52s   172.31.30.106   ip-172-31-30-106   <none>           <none>
kube-system       pod/rke2-multus-p5vr7                                       1/1     Running     3 (74s ago)     116s    172.31.24.197   ip-172-31-24-197   <none>           <none>
kube-system       pod/rke2-snapshot-controller-59cc9cd8f4-bljlp               1/1     Running     0               118s    10.42.17.140    ip-172-31-30-106   <none>           <none>
kube-system       pod/rke2-snapshot-validation-webhook-54c5989b65-dgfkt       1/1     Running     0               2m4s    10.42.17.138    ip-172-31-30-106   <none>           <none>
tigera-operator   pod/tigera-operator-59d6c9b46-n8fmw                         1/1     Running     0               3m31s   172.31.30.106   ip-172-31-30-106   <none>           <none>

$ ls -l /run/cni/
total 0
srwxr-xr-x 1 root root 0 Jan 24 17:47 dhcp.sock

Additional context / logs:

N/A

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees

@endawkins endawkins

Labels
None yet
Projects
None yet
Milestone
v1.28.6+rke2r1
Development

No branches or pull requests
2 participants
@thomasferrandiz @endawkins