k3s air-gapped env resolve dns fails for statefulset #11343
-
|
An air-gapped env has been configured with one master and a worker. k3sk3s version v1.30.3+k3s1 (f6466040)
go version go1.22.5osPRETTY_NAME="Debian GNU/Linux 12 (bookworm)"
NAME="Debian GNU/Linux"
VERSION_ID="12"
VERSION="12 (bookworm)"
VERSION_CODENAME=bookworm
ID=debian
HOME_URL="https://www.debian.org/"
SUPPORT_URL="https://www.debian.org/support"
BUG_REPORT_URL="https://bugs.debian.org/"master#!/bin/bash
set -ex
cp -v ./k3s /usr/local/bin/
export INSTALL_K3S_SKIP_DOWNLOAD=true
export INSTALL_K3S_EXEC="--disable=traefik --cluster-cidr=10.42.0.0/16 --service-cidr=10.43.0.0/16 --node-external-ip=172.16.110.124 --node-ip=172.16.110.124 --node-label=100m1"
./install.shworker#!/bin/bash
set -ex
cp -v ./k3s /usr/local/bin/
export INSTALL_K3S_SKIP_DOWNLOAD=true
export K3S_URL=https://172.16.110.124:6443
export INSTALL_K3S_EXEC="--node-external-ip=172.16.110.126 --node-ip=172.16.110.126"
export K3S_TOKEN='xyz'
./install.shsample podTwo pods have been created and set on a single node apiVersion: v1
kind: Service
metadata:
name: nginx
labels:
app: nginx
spec:
ports:
- port: 80
name: web
clusterIP: None
selector:
app: nginx
---
apiVersion: apps/v1
kind: StatefulSet
metadata:
name: web
labels:
app: nginx
spec:
serviceName: "nginx"
replicas: 2
selector:
matchLabels:
app: nginx
template:
metadata:
labels:
app: nginx
spec:
# hostname: nginx
subdomain: nginx # to resolve dns
nodeName: 100w3 # <-- the worker
containers:
- name: nginx
image: nginx:stable-alpine3.20
ports:
- containerPort: 80
name: web
volumeMounts:
- name: www
mountPath: /usr/share/nginx/html
volumes:
- name: www
emptyDir: {}issueOn this air-gapped env web-0 and web-1 cannot see (
testsOut of the air-gapped env, (public access to Internet) with the exact same k3s version and nginx config it works fine. questionIs this issue because of air-gapped env ? screenshotsair-gapped
public more infoNothing has been blocked on machines, all ports and protos are allowed . Using kubectl get svc kubectl get ep kubectl get ep -n kube-system kubectl get pods -n kube-system crictl image ls both web-0 and web-1 can Does anyone have any idea ? Regards |
Beta Was this translation helpful? Give feedback.
Replies: 3 comments 7 replies
-
|
Perhaps https://docs.k3s.io/installation/airgap#default-network-route |
Beta Was this translation helpful? Give feedback.
-
|
I do not think so, each node has its interface and its default route and they have been configured and can see each other |
Beta Was this translation helpful? Give feedback.
-
Is the default route associated with the interface the nodes use to communicate with each other? Note that you're testing dns resolution with |
Beta Was this translation helpful? Give feedback.
-
|
You disabled traefik, what are you using as its replacement? |
Beta Was this translation helpful? Give feedback.
-
|
nothing, it is a test and I did not need the ingress-controller |
Beta Was this translation helpful? Give feedback.
-
|
kubectl get nodes -o wide Also I tried other setups, like no external-ip, but same result masterip route show ip -br a workerip route show ip -br a You can see Docker routers here , since I installed and tested Docker Swarm to test if the issue is by the Machines or not. Docker Swarm DNS resolution works fine |
Beta Was this translation helpful? Give feedback.
-
For K3s, you should disable on flannel.1 instead of flannel.4096 |
Beta Was this translation helpful? Give feedback.
-
|
the work around solved and showed this was the cause , thank you |
Beta Was this translation helpful? Give feedback.
-
|
You said it worked outside your airgapped environment, but I'm guessing the difference here was use of vmware, not airgap vs connected to internet. Its important to identify ALL the differences between environments, and not get too focused on troubleshooting one thing. |
Beta Was this translation helpful? Give feedback.
-
Yes , the VMs outside were KVM machines working machinesHypervisor vendor: KVMair-gapped not workingHypervisor vendor: VMware |
Beta Was this translation helpful? Give feedback.
For K3s, you should disable on flannel.1 instead of flannel.4096