Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

[release-1.26] DaemonSet for ContainerNetworking DHCP CNI Plugin #5209

Closed
thomasferrandiz opened this issue Jan 5, 2024 · 1 comment
Closed

[release-1.26] DaemonSet for ContainerNetworking DHCP CNI Plugin #5209

thomasferrandiz opened this issue Jan 5, 2024 · 1 comment
Assignees
@endawkins
Milestone
v1.26.13+rke2r1

Comments

@thomasferrandiz
Copy link
Contributor

Backport of: #3917
@thomasferrandiz thomasferrandiz added this to the v1.26.13+rke2r1 milestone Jan 5, 2024
@thomasferrandiz thomasferrandiz mentioned this issue Jan 5, 2024
Merged
@endawkins
Copy link

Validated on 1.26 with 759709e / version 1.26

Environment Details

Infrastructure

  • Cloud
  • Hosted

Node(s) CPU architecture, OS, and Version:

Linux ip-172-31-24-102 6.2.0-1017-aws #17~22.04.1-Ubuntu SMP Fri Nov 17 21:07:13 UTC 2023 x86_64 x86_64 x86_64 GNU/Linux
PRETTY_NAME="Ubuntu 22.04.3 LTS"
NAME="Ubuntu"
VERSION_ID="22.04"
VERSION="22.04.3 LTS (Jammy Jellyfish)"
VERSION_CODENAME=jammy
ID=ubuntu
ID_LIKE=debian
HOME_URL="https://www.ubuntu.com/"
SUPPORT_URL="https://help.ubuntu.com/"
BUG_REPORT_URL="https://bugs.launchpad.net/ubuntu/"
PRIVACY_POLICY_URL="https://www.ubuntu.com/legal/terms-and-policies/privacy-policy"
UBUNTU_CODENAME=jammy

Cluster Configuration:

1 server
1 agent

Config.yaml:

write-kubeconfig-mode: 644
token: test
node-external-ip: <EXTERNAL_IP>
cni: multus,calico

Additional files

***rke2-multus-config.yaml

apiVersion: helm.cattle.io/v1
kind: HelmChartConfig
metadata:
  name: rke2-multus
  namespace: kube-system
spec:
  valuesContent: |-
    manifests:
      dhcpDaemonSet: true

Testing Steps

  1. Copy config.yaml
$ sudo mkdir -p /etc/rancher/rke2 && sudo cp config.yaml /etc/rancher/rke2
  1. Install RKE2 (Do not enable or start rke2)
  2. Create file /var/lib/rancher/rke2/server/manifests/rke2-multus-config.yaml
  3. Enable and start rke2 (both server and agent)
  4. Verify that kube-rke2-multus-dhcp was created and started properly
  5. Verify /run/cni/dhcp.sock was created on agent node

Replication Results:

  • rke2 version used for replication:
$ rke2 -v
rke2 version v1.26.12+rke2r1 (10683d5a09610873f643ada416b97eb01859108d)
go version go1.20.12 X:boringcrypto

$ kubectl get nodes,pods -A -o wide                                                                                                                                                       ip-172-31-24-102: Wed Jan 24 19:49:20 2024

NAME                    STATUS   ROLES                       AGE   VERSION           INTERNAL-IP     EXTERNAL-IP     OS-IMAGE             KERNEL-VERSION   CONTAINER-RUNTIME
node/ip-172-31-22-131   Ready    <none>                      10m   v1.26.12+rke2r1   172.31.22.131   <none>          Ubuntu 22.04.3 LTS   6.2.0-1017-aws   containerd://1.7.11-k3s2
node/ip-172-31-24-102   Ready    control-plane,etcd,master   12m   v1.26.12+rke2r1   172.31.24.102   <EXTERNAL_IP>   Ubuntu 22.04.3 LTS   6.2.0-1017-aws   containerd://1.7.11-k3s2

NAMESPACE         NAME                                                        READY   STATUS      RESTARTS      AGE   IP              NODE               NOMINATED NODE   READINESS GATES
calico-system     pod/calico-kube-controllers-585577645b-rkpm9                1/1     Running     0             11m   10.42.13.130    ip-172-31-24-102   <none>           <none>
calico-system     pod/calico-node-rtpph                                       1/1     Running     0             10m   172.31.22.131   ip-172-31-22-131   <none>           <none>
calico-system     pod/calico-node-t6vz5                                       1/1     Running     0             11m   172.31.24.102   ip-172-31-24-102   <none>           <none>
calico-system     pod/calico-typha-ddcc977dd-kdfhb                            1/1     Running     0             11m   172.31.24.102   ip-172-31-24-102   <none>           <none>
kube-system       pod/cloud-controller-manager-ip-172-31-24-102               1/1     Running     0             12m   172.31.24.102   ip-172-31-24-102   <none>           <none>
kube-system       pod/etcd-ip-172-31-24-102                                   1/1     Running     0             12m   172.31.24.102   ip-172-31-24-102   <none>           <none>
kube-system       pod/helm-install-rke2-calico-6hb5l                          0/1     Completed   2             11m   172.31.24.102   ip-172-31-24-102   <none>           <none>
kube-system       pod/helm-install-rke2-calico-crd-h57b7                      0/1     Completed   0             11m   172.31.24.102   ip-172-31-24-102   <none>           <none>
kube-system       pod/helm-install-rke2-coredns-49ww2                         0/1     Completed   0             11m   172.31.24.102   ip-172-31-24-102   <none>           <none>
kube-system       pod/helm-install-rke2-ingress-nginx-jmjwg                   0/1     Completed   0             11m   10.42.13.134    ip-172-31-24-102   <none>           <none>
kube-system       pod/helm-install-rke2-metrics-server-jprdv                  0/1     Completed   0             11m   10.42.13.132    ip-172-31-24-102   <none>           <none>
kube-system       pod/helm-install-rke2-multus-gpwfn                          0/1     Completed   0             11m   172.31.24.102   ip-172-31-24-102   <none>           <none>
kube-system       pod/helm-install-rke2-snapshot-controller-8nrlp             0/1     Completed   0             11m   10.42.13.133    ip-172-31-24-102   <none>           <none>
kube-system       pod/helm-install-rke2-snapshot-controller-crd-qzhp8         0/1     Completed   0             11m   10.42.13.129    ip-172-31-24-102   <none>           <none>
kube-system       pod/helm-install-rke2-snapshot-validation-webhook-p4pj4     0/1     Completed   0             11m   10.42.13.131    ip-172-31-24-102   <none>           <none>
kube-system       pod/kube-apiserver-ip-172-31-24-102                         1/1     Running     0             12m   172.31.24.102   ip-172-31-24-102   <none>           <none>
kube-system       pod/kube-controller-manager-ip-172-31-24-102                1/1     Running     0             12m   172.31.24.102   ip-172-31-24-102   <none>           <none>
kube-system       pod/kube-proxy-ip-172-31-22-131                             1/1     Running     0             10m   172.31.22.131   ip-172-31-22-131   <none>           <none>
kube-system       pod/kube-proxy-ip-172-31-24-102                             1/1     Running     0             12m   172.31.24.102   ip-172-31-24-102   <none>           <none>
kube-system       pod/kube-scheduler-ip-172-31-24-102                         1/1     Running     0             12m   172.31.24.102   ip-172-31-24-102   <none>           <none>
kube-system       pod/rke2-coredns-rke2-coredns-565dfc7d75-954ss              1/1     Running     0             11m   10.42.13.136    ip-172-31-24-102   <none>           <none>
kube-system       pod/rke2-coredns-rke2-coredns-565dfc7d75-9j9jw              1/1     Running     0             10m   10.42.30.2      ip-172-31-22-131   <none>           <none>
kube-system       pod/rke2-coredns-rke2-coredns-autoscaler-6c48c95bf9-czs57   1/1     Running     0             11m   10.42.13.135    ip-172-31-24-102   <none>           <none>
kube-system       pod/rke2-ingress-nginx-controller-7k89h                     1/1     Running     0             10m   10.42.13.141    ip-172-31-24-102   <none>           <none>
kube-system       pod/rke2-ingress-nginx-controller-82hwm                     1/1     Running     0             10m   10.42.30.1      ip-172-31-22-131   <none>           <none>
kube-system       pod/rke2-metrics-server-c9c78bd66-vtqbx                     1/1     Running     0             10m   10.42.13.139    ip-172-31-24-102   <none>           <none>
kube-system       pod/rke2-multus-ds-j7874                                    1/1     Running     0             10m   172.31.22.131   ip-172-31-22-131   <none>           <none>
kube-system       pod/rke2-multus-ds-sd6qf                                    1/1     Running     3 (11m ago)   11m   172.31.24.102   ip-172-31-24-102   <none>           <none>
kube-system       pod/rke2-snapshot-controller-6f7bbb497d-tn86c               1/1     Running     0             10m   10.42.13.138    ip-172-31-24-102   <none>           <none>
kube-system       pod/rke2-snapshot-validation-webhook-65b5675d5c-qt2v7       1/1     Running     0             10m   10.42.13.137    ip-172-31-24-102   <none>           <none>
tigera-operator   pod/tigera-operator-767d6945f4-zl585                        1/1     Running     0             11m   172.31.24.102   ip-172-31-24-102   <none>           <none>

$ ls -l /run/cni/
ls: cannot access '/run/cni/': No such file or directory

Validation Results:

  • rke2 version used for validation:
$ rke2 -v
rke2 version v1.26.13-rc2+rke2r1 (759709e78f0b5138a2d632aa5665d2b2c5dcdc10)
go version go1.20.13 X:boringcrypto

$ kubectl get nodes,pods -A -o wide                                                                                                                                                       ip-172-31-24-102: Wed Jan 24 20:02:32 2024

NAME                    STATUS   ROLES                       AGE    VERSION           INTERNAL-IP     EXTERNAL-IP     OS-IMAGE             KERNEL-VERSION   CONTAINER-RUNTIME
node/ip-172-31-22-131   Ready    <none>                      66s    v1.26.13+rke2r1   172.31.22.131   <none>          Ubuntu 22.04.3 LTS   6.2.0-1017-aws   containerd://1.7.11-k3s2
node/ip-172-31-24-102   Ready    control-plane,etcd,master   4m2s   v1.26.13+rke2r1   172.31.24.102   3.145.198.195   Ubuntu 22.04.3 LTS   6.2.0-1017-aws   containerd://1.7.11-k3s2

NAMESPACE         NAME                                                       READY   STATUS      RESTARTS       AGE     IP              NODE               NOMINATED NODE   READINESS GATES
calico-system     pod/calico-kube-controllers-6596c6b6d7-8mj27               1/1     Running     0              3m14s   10.42.13.133    ip-172-31-24-102   <none>           <none>
calico-system     pod/calico-node-524pc                                      1/1     Running     0              3m14s   172.31.24.102   ip-172-31-24-102   <none>           <none>
calico-system     pod/calico-node-flltx                                      1/1     Running     0              66s     172.31.22.131   ip-172-31-22-131   <none>           <none>
calico-system     pod/calico-typha-6b866b7769-psjlx                          1/1     Running     0              3m14s   172.31.24.102   ip-172-31-24-102   <none>           <none>
kube-system       pod/cloud-controller-manager-ip-172-31-24-102              1/1     Running     0              3m55s   172.31.24.102   ip-172-31-24-102   <none>           <none>
kube-system       pod/etcd-ip-172-31-24-102                                  1/1     Running     0              3m40s   172.31.24.102   ip-172-31-24-102   <none>           <none>
kube-system       pod/helm-install-rke2-calico-272cb                         0/1     Completed   2              3m47s   172.31.24.102   ip-172-31-24-102   <none>           <none>
kube-system       pod/helm-install-rke2-calico-crd-2n64t                     0/1     Completed   0              3m47s   172.31.24.102   ip-172-31-24-102   <none>           <none>
kube-system       pod/helm-install-rke2-coredns-tbp5m                        0/1     Completed   0              3m47s   172.31.24.102   ip-172-31-24-102   <none>           <none>
kube-system       pod/helm-install-rke2-ingress-nginx-22n2c                  0/1     Completed   0              3m47s   10.42.13.129    ip-172-31-24-102   <none>           <none>
kube-system       pod/helm-install-rke2-metrics-server-nj2nm                 0/1     Completed   0              3m47s   10.42.13.136    ip-172-31-24-102   <none>           <none>
kube-system       pod/helm-install-rke2-multus-lx6w7                         0/1     Completed   0              3m47s   172.31.24.102   ip-172-31-24-102   <none>           <none>
kube-system       pod/helm-install-rke2-snapshot-controller-crd-s4crt        0/1     Completed   0              3m47s   10.42.13.134    ip-172-31-24-102   <none>           <none>
kube-system       pod/helm-install-rke2-snapshot-controller-d6rlp            0/1     Completed   1              3m47s   10.42.13.135    ip-172-31-24-102   <none>           <none>
kube-system       pod/helm-install-rke2-snapshot-validation-webhook-b79wc    0/1     Completed   0              3m47s   10.42.13.130    ip-172-31-24-102   <none>           <none>
kube-system       pod/kube-apiserver-ip-172-31-24-102                        1/1     Running     0              3m55s   172.31.24.102   ip-172-31-24-102   <none>           <none>
kube-system       pod/kube-controller-manager-ip-172-31-24-102               1/1     Running     0              4m1s    172.31.24.102   ip-172-31-24-102   <none>           <none>
kube-system       pod/kube-proxy-ip-172-31-22-131                            1/1     Running     0              65s     172.31.22.131   ip-172-31-22-131   <none>           <none>
kube-system       pod/kube-proxy-ip-172-31-24-102                            1/1     Running     0              3m58s   172.31.24.102   ip-172-31-24-102   <none>           <none>
kube-system       pod/kube-scheduler-ip-172-31-24-102                        1/1     Running     0              4m1s    172.31.24.102   ip-172-31-24-102   <none>           <none>
kube-system       pod/rke2-coredns-rke2-coredns-7df9d89b69-5hz26             0/1     Running     0              60s     10.42.30.1      ip-172-31-22-131   <none>           <none>
kube-system       pod/rke2-coredns-rke2-coredns-7df9d89b69-cmftv             1/1     Running     0              3m37s   10.42.13.132    ip-172-31-24-102   <none>           <none>
kube-system       pod/rke2-coredns-rke2-coredns-autoscaler-6c794d698-r249k   1/1     Running     0              3m37s   10.42.13.131    ip-172-31-24-102   <none>           <none>
kube-system       pod/rke2-ingress-nginx-controller-rljpb                    1/1     Running     0              2m26s   10.42.13.140    ip-172-31-24-102   <none>           <none>
kube-system       pod/rke2-ingress-nginx-controller-vwvwv                    0/1     Running     0              39s     10.42.30.0      ip-172-31-22-131   <none>           <none>
kube-system       pod/rke2-metrics-server-84b8d89697-9k4vp                   1/1     Running     0              2m29s   10.42.13.139    ip-172-31-24-102   <none>           <none>
kube-system       pod/rke2-multus-btpch                                      1/1     Running     1 (39s ago)    66s     172.31.22.131   ip-172-31-22-131   <none>           <none>
kube-system       pod/rke2-multus-dhcp-wps8x                                 1/1     Running     0              3m2s    172.31.24.102   ip-172-31-24-102   <none>           <none>
kube-system       pod/rke2-multus-dhcp-zpfxw                                 1/1     Running     0              39s     172.31.22.131   ip-172-31-22-131   <none>           <none>
kube-system       pod/rke2-multus-wpvk4                                      1/1     Running     3 (3m5s ago)   3m38s   172.31.24.102   ip-172-31-24-102   <none>           <none>
kube-system       pod/rke2-snapshot-controller-6f7bbb497d-fbnwd              1/1     Running     0              2m23s   10.42.13.142    ip-172-31-24-102   <none>           <none>
kube-system       pod/rke2-snapshot-validation-webhook-65b5675d5c-4tmr5      1/1     Running     0              2m33s   10.42.13.138    ip-172-31-24-102   <none>           <none>
tigera-operator   pod/tigera-operator-767d6945f4-wm96z                       1/1     Running     0              3m20s   172.31.24.102   ip-172-31-24-102   <none>           <none>

$ ls -l /run/cni/
total 0
srwxr-xr-x 1 root root 0 Jan 24 20:01 dhcp.sock

Additional context / logs:

N/A

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees

@endawkins endawkins

Labels
None yet
Projects
None yet
Milestone
v1.26.13+rke2r1
Development

No branches or pull requests
2 participants
@thomasferrandiz @endawkins