Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

[Release-1.26] - Calico pollutes the rke2-win logs #4789

Closed
manuelbuil opened this issue Sep 25, 2023 · 1 comment
Closed

[Release-1.26] - Calico pollutes the rke2-win logs #4789

manuelbuil opened this issue Sep 25, 2023 · 1 comment
Assignees
@VestigeJ @manuelbuil
Milestone
v1.26.10+rke2r1

Comments

@manuelbuil
Copy link
Contributor

Backport fix for Calico pollutes the rke2-win logs
@manuelbuil manuelbuil self-assigned this Sep 25, 2023
@manuelbuil manuelbuil mentioned this issue Sep 25, 2023
Merged
@manuelbuil manuelbuil added this to the v1.26.10+rke2r1 milestone Sep 25, 2023
@VestigeJ
Copy link
Contributor

Reproduced here #4722 (comment)

Validated logs are cleaned up

$ kubectl get nodes

NAME                    STATUS   ROLES                       AGE     VERSION
node/ec2amaz-hm07pcb    Ready    <none>                      2m12s   v1.26.10
node/ip-1-1-1-23        Ready    control-plane,etcd,master   16m     v1.26.10+rke2r1
node/ip-1-1-1-16        Ready    <none>                      14m     v1.26.10+rke2r1

$ get-eventlog -logname Application -source 'rke2' | select-object -Property TimeWritten,ReplacementStrings | format-table -wrap

TimeWritten            ReplacementStrings
-----------            ------------------
10/25/2023 10:18:32 PM {Running RKE2 kube-proxy [--bind-address=172.31.20.251 --enable-dsr=true --feature-gates=WinDSR=true --network-name=Calico
                       --source-vip=10.42.116.2 --cluster-cidr=10.42.0.0/16 --healthz-bind-address=127.0.0.1 --hostname-override=ec2amaz-hm07pcb
                       --kubeconfig=C:\var\lib\rancher\rke2\agent\kubeproxy.kubeconfig --proxy-mode=kernelspace]}
10/25/2023 10:18:32 PM {Deleting HNS policies before kube-proxy starts.}
10/25/2023 10:18:27 PM {Felix Envs: [KUBE_NETWORK=Calico.* KUBECONFIG=c:\var\lib\rancher\rke2\agent\calico.kubeconfig NODENAME=ec2amaz-hm07pcb
                       CALICO_K8S_NODE_REF=ec2amaz-hm07pcb IP=172.31.20.251 USE_POD_CIDR=false FELIX_FELIXHOSTNAME=ec2amaz-hm07pcb
                       FELIX_VXLANVNI=4096 FELIX_DATASTORETYPE=kubernetes]}
10/25/2023 10:18:27 PM {can't find Calico_ep HNS endpoint, retrying, error=Endpoint Calico_ep not found}
10/25/2023 10:18:23 PM {can't find Calico_ep HNS endpoint, retrying, error=Endpoint Calico_ep not found}
10/25/2023 10:18:23 PM {Calico Envs: [KUBE_NETWORK=Calico.* KUBECONFIG=c:\var\lib\rancher\rke2\agent\calico.kubeconfig NODENAME=ec2amaz-hm07pcb
                       CALICO_K8S_NODE_REF=ec2amaz-hm07pcb IP=172.31.20.251 USE_POD_CIDR=false
                       CALICO_NODENAME_FILE=c:\var\lib\rancher\rke2\agent\calico_node_name CALICO_NETWORKING_BACKEND=vxlan
                       CALICO_DATASTORE_TYPE=kubernetes IP_AUTODETECTION_METHOD=first-found VXLAN_VNI=4096]}
10/25/2023 10:18:23 PM {Calico exited: exit status 1. Retrying}
10/25/2023 10:18:20 PM {Tunnel authorizer set Kubelet Port 10250}
10/25/2023 10:18:20 PM {Annotations and labels have been set successfully on node: ec2amaz-hm07pcb}
10/25/2023 10:18:20 PM {Failed to set annotations and labels on node ec2amaz-hm07pcb: Operation cannot be fulfilled on nodes "ec2amaz-hm07pcb": the
                       object has been modified; please apply your changes to the latest version and try again}
10/25/2023 10:18:20 PM {Failed to set annotations and labels on node ec2amaz-hm07pcb: Operation cannot be fulfilled on nodes "ec2amaz-hm07pcb": the
                       object has been modified; please apply your changes to the latest version and try again}
10/25/2023 10:18:20 PM {Failed to set annotations and labels on node ec2amaz-hm07pcb: Operation cannot be fulfilled on nodes "ec2amaz-hm07pcb": the
                       object has been modified; please apply your changes to the latest version and try again}
10/25/2023 10:18:17 PM {WinDSR support is enabled}
10/25/2023 10:18:17 PM {HCN feature check, version={13 3} supportedFeatures={{true true true true} {true true} true true true true true true true
                       true true true true false false false}}
10/25/2023 10:18:17 PM {Running kube-proxy --cluster-cidr=10.42.0.0/16 --healthz-bind-address=127.0.0.1 --hostname-override=ec2amaz-hm07pcb
                       --kubeconfig=C:\var\lib\rancher\rke2\agent\kubeproxy.kubeconfig --proxy-mode=kernelspace}
10/25/2023 10:18:16 PM {Calico Envs: [KUBE_NETWORK=Calico.* KUBECONFIG=c:\var\lib\rancher\rke2\agent\calico.kubeconfig NODENAME=ec2amaz-hm07pcb
                       CALICO_K8S_NODE_REF=ec2amaz-hm07pcb IP=172.31.20.251 USE_POD_CIDR=false
                       CALICO_NODENAME_FILE=c:\var\lib\rancher\rke2\agent\calico_node_name CALICO_NETWORKING_BACKEND=vxlan
                       CALICO_DATASTORE_TYPE=kubernetes IP_AUTODETECTION_METHOD=first-found VXLAN_VNI=4096]}
10/25/2023 10:18:16 PM {Running RKE2 kubelet [--cgroups-per-qos=false --enforce-node-allocatable= --file-check-frequency=5s
                       --hairpin-mode=promiscuous-bridge --resolv-conf= --sync-frequency=30s --address=0.0.0.0 --anonymous-auth=false
                       --authentication-token-webhook=true --authorization-mode=Webhook
                       --client-ca-file=C:\var\lib\rancher\rke2\agent\client-ca.crt --cloud-provider=external --cluster-dns=10.43.0.10
                       --cluster-domain=cluster.local --container-runtime-endpoint=npipe:////./pipe/containerd-containerd --fail-swap-on=false
                       --healthz-bind-address=127.0.0.1 --hostname-override=ec2amaz-hm07pcb
                       --kubeconfig=C:\var\lib\rancher\rke2\agent\kubelet.kubeconfig --node-labels=
                       --pod-infra-container-image=index.docker.io/rancher/pause:3.6
                       --pod-manifest-path=C:\var\lib\rancher\rke2\agent\pod-manifests --read-only-port=0
                       --resolv-conf=C:\var\lib\rancher\rke2\agent\etc\resolv.conf --serialize-image-pulls=false
                       --tls-cert-file=C:\var\lib\rancher\rke2\agent\serving-kubelet.crt
                       --tls-private-key-file=C:\var\lib\rancher\rke2\agent\serving-kubelet.key]}
10/25/2023 10:18:16 PM {Running kubelet --address=0.0.0.0 --alsologtostderr=false --anonymous-auth=false --authentication-token-webhook=true
                       --authorization-mode=Webhook --client-ca-file=C:\var\lib\rancher\rke2\agent\client-ca.crt --cloud-provider=external
                       --cluster-dns=10.43.0.10 --cluster-domain=cluster.local --container-runtime-endpoint=npipe:////./pipe/containerd-containerd
                       --eviction-hard=imagefs.available<5%,nodefs.available<5%
                       --eviction-minimum-reclaim=imagefs.available=10%,nodefs.available=10% --fail-swap-on=false --healthz-bind-address=127.0.0.1
                       --hostname-override=ec2amaz-hm07pcb --kubeconfig=C:\var\lib\rancher\rke2\agent\kubelet.kubeconfig
                       --log-file=\var\lib\rancher\rke2\agent\logs\kubelet.log --log-file-max-size=50 --logtostderr=false --node-labels=
                       --pod-infra-container-image=index.docker.io/rancher/pause:3.6
                       --pod-manifest-path=C:\var\lib\rancher\rke2\agent\pod-manifests --read-only-port=0
                       --resolv-conf=C:\var\lib\rancher\rke2\agent\etc\resolv.conf --serialize-image-pulls=false --stderrthreshold=FATAL
                       --tls-cert-file=C:\var\lib\rancher\rke2\agent\serving-kubelet.crt
                       --tls-private-key-file=C:\var\lib\rancher\rke2\agent\serving-kubelet.key}
10/25/2023 10:18:16 PM {Connecting to proxy, url=wss://1.1.1.17:9345/v1-rke2/connect}
10/25/2023 10:18:16 PM {Updated load balancer rke2-agent-load-balancer default server address -> 1.1.1.17:9345}
10/25/2023 10:18:16 PM {Getting list of apiserver endpoints from server}
10/25/2023 10:18:16 PM {containerd is now running}
10/25/2023 10:18:15 PM {Running containerd -c C:\var\lib\rancher\rke2\agent\etc\containerd\config.toml}
10/25/2023 10:18:15 PM {Logging containerd to C:\var\lib\rancher\rke2\agent\containerd\containerd.log}
10/25/2023 10:18:15 PM {Windows bootstrap okay. Exiting setup.}
10/25/2023 10:18:09 PM {Creating VXLAN network using the vxlanAdapter: Ethernet 3}
10/25/2023 10:18:09 PM {Generating HNS networks, please wait}
10/25/2023 10:18:07 PM {Extracting file bin\ctr.exe to C:\var\lib\rancher\rke2\data\v1.26.10-rc2-rke2r1-windows-amd64-1d1bf450887b\bin\ctr.exe}
10/25/2023 10:18:06 PM {Extracting file bin\containerd.exe to
                       C:\var\lib\rancher\rke2\data\v1.26.10-rc2-rke2r1-windows-amd64-1d1bf450887b\bin\containerd.exe}
10/25/2023 10:18:06 PM {Extracting file bin\containerd-stress.exe to
                       C:\var\lib\rancher\rke2\data\v1.26.10-rc2-rke2r1-windows-amd64-1d1bf450887b\bin\containerd-stress.exe}
10/25/2023 10:18:05 PM {Extracting file bin\win-overlay.exe to
                       C:\var\lib\rancher\rke2\data\v1.26.10-rc2-rke2r1-windows-amd64-1d1bf450887b\bin\win-overlay.exe}
10/25/2023 10:18:04 PM {Extracting file bin\kubelet.exe to
                       C:\var\lib\rancher\rke2\data\v1.26.10-rc2-rke2r1-windows-amd64-1d1bf450887b\bin\kubelet.exe}
10/25/2023 10:18:03 PM {Extracting file bin\kubectl.exe to
                       C:\var\lib\rancher\rke2\data\v1.26.10-rc2-rke2r1-windows-amd64-1d1bf450887b\bin\kubectl.exe}
10/25/2023 10:18:02 PM {Extracting file bin\kube-proxy.exe to
                       C:\var\lib\rancher\rke2\data\v1.26.10-rc2-rke2r1-windows-amd64-1d1bf450887b\bin\kube-proxy.exe}
10/25/2023 10:18:02 PM {Extracting file bin\host-local.exe to
                       C:\var\lib\rancher\rke2\data\v1.26.10-rc2-rke2r1-windows-amd64-1d1bf450887b\bin\host-local.exe}
10/25/2023 10:18:02 PM {Extracting file bin\hns.psm1 to C:\var\lib\rancher\rke2\data\v1.26.10-rc2-rke2r1-windows-amd64-1d1bf450887b\bin\hns.psm1}
10/25/2023 10:18:01 PM {Extracting file bin\crictl.exe to
                       C:\var\lib\rancher\rke2\data\v1.26.10-rc2-rke2r1-windows-amd64-1d1bf450887b\bin\crictl.exe}
10/25/2023 10:18:01 PM {Extracting file bin\containerd-shim-runhcs-v1.exe to
                       C:\var\lib\rancher\rke2\data\v1.26.10-rc2-rke2r1-windows-amd64-1d1bf450887b\bin\containerd-shim-runhcs-v1.exe}
10/25/2023 10:18:00 PM {Extracting file bin\calico.exe to
                       C:\var\lib\rancher\rke2\data\v1.26.10-rc2-rke2r1-windows-amd64-1d1bf450887b\bin\calico.exe}
10/25/2023 10:17:59 PM {Extracting file bin\calico-node.exe to
                       C:\var\lib\rancher\rke2\data\v1.26.10-rc2-rke2r1-windows-amd64-1d1bf450887b\bin\calico-node.exe}
10/25/2023 10:17:58 PM {Extracting file bin\calico-ipam.exe to
                       C:\var\lib\rancher\rke2\data\v1.26.10-rc2-rke2r1-windows-amd64-1d1bf450887b\bin\calico-ipam.exe}
10/25/2023 10:17:58 PM {Extracting file bin\confd\templates\peerings.ps1.template to
                       C:\var\lib\rancher\rke2\data\v1.26.10-rc2-rke2r1-windows-amd64-1d1bf450887b\bin\confd\templates\peerings.ps1.template}
10/25/2023 10:17:58 PM {Extracting file bin\confd\templates\blocks.ps1.template to
                       C:\var\lib\rancher\rke2\data\v1.26.10-rc2-rke2r1-windows-amd64-1d1bf450887b\bin\confd\templates\blocks.ps1.template}
10/25/2023 10:17:58 PM {Creating directory C:\var\lib\rancher\rke2\data\v1.26.10-rc2-rke2r1-windows-amd64-1d1bf450887b\bin\confd\templates}
10/25/2023 10:17:57 PM {Extracting file bin\confd\config-bgp.psm1 to
                       C:\var\lib\rancher\rke2\data\v1.26.10-rc2-rke2r1-windows-amd64-1d1bf450887b\bin\confd\config-bgp.psm1}
10/25/2023 10:17:57 PM {Extracting file bin\confd\config-bgp.ps1 to
                       C:\var\lib\rancher\rke2\data\v1.26.10-rc2-rke2r1-windows-amd64-1d1bf450887b\bin\confd\config-bgp.ps1}
10/25/2023 10:17:57 PM {Extracting file bin\confd\confd-service.ps1 to
                       C:\var\lib\rancher\rke2\data\v1.26.10-rc2-rke2r1-windows-amd64-1d1bf450887b\bin\confd\confd-service.ps1}
10/25/2023 10:17:57 PM {Extracting file bin\confd\conf.d\peerings.toml to
                       C:\var\lib\rancher\rke2\data\v1.26.10-rc2-rke2r1-windows-amd64-1d1bf450887b\bin\confd\conf.d\peerings.toml}
10/25/2023 10:17:57 PM {Extracting file bin\confd\conf.d\blocks.toml to
                       C:\var\lib\rancher\rke2\data\v1.26.10-rc2-rke2r1-windows-amd64-1d1bf450887b\bin\confd\conf.d\blocks.toml}
10/25/2023 10:17:57 PM {Creating directory C:\var\lib\rancher\rke2\data\v1.26.10-rc2-rke2r1-windows-amd64-1d1bf450887b\bin\confd\conf.d}
10/25/2023 10:17:57 PM {Creating directory C:\var\lib\rancher\rke2\data\v1.26.10-rc2-rke2r1-windows-amd64-1d1bf450887b\bin\confd}
10/25/2023 10:17:57 PM {Creating directory C:\var\lib\rancher\rke2\data\v1.26.10-rc2-rke2r1-windows-amd64-1d1bf450887b\bin}
10/25/2023 10:17:56 PM {Pulling runtime image index.docker.io/rancher/rke2-runtime:v1.26.10-rc2-rke2r1-windows-amd64}
10/25/2023 10:17:56 PM {Failed to load runtime image index.docker.io/rancher/rke2-runtime:v1.26.10-rc2-rke2r1-windows-amd64 from tarball: no local
                       image available for index.docker.io/rancher/rke2-runtime:v1.26.10-rc2-rke2r1-windows-amd64: directory
                       C:\var\lib\rancher\rke2\agent\images does not exist: image not found}
10/25/2023 10:17:56 PM {Failed to load runtime image index.docker.io/rancher/rke2-runtime:v1.26.10-rc2-rke2r1-windows-amd64 from tarball: no local
                       image available for index.docker.io/rancher/rke2-runtime:v1.26.10-rc2-rke2r1-windows-amd64: directory
                       C:\var\lib\rancher\rke2\agent\images does not exist: image not found}
10/25/2023 10:17:56 PM {Host resolv.conf includes loopback or multicast nameservers - kubelet will use autogenerated resolv.conf with nameserver
                       8.8.8.8}
10/25/2023 10:17:55 PM {Running load balancer rke2-api-server-agent-load-balancer 127.0.0.1:6443 -> [1.1.1.17:6443] [default:
                       1.1.1.17:6443]}
10/25/2023 10:17:55 PM {Adding server to load balancer rke2-api-server-agent-load-balancer: 1.1.1.17:6443}
10/25/2023 10:17:54 PM {Running load balancer rke2-agent-load-balancer 127.0.0.1:6444 -> [1.1.1.17:9345] [default: 1.1.1.17:9345]}
10/25/2023 10:17:54 PM {Adding server to load balancer rke2-agent-load-balancer: 1.1.1.17:9345}
10/25/2023 10:17:54 PM {Starting rke2 agent v1.26.10-rc2+rke2r1 (825e3188d273e7271a0b5ce924d42455b4d37a34)}
10/25/2023 10:17:54 PM {Stackdump - waiting signal at Global\stackdump-2760}

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees

@VestigeJ VestigeJ

@manuelbuil manuelbuil

Labels
None yet
Projects
None yet
Milestone
v1.26.10+rke2r1
Development

No branches or pull requests
2 participants
@VestigeJ @manuelbuil