[Release-1.27] - DNS over TLS not working

[manuelbuil]

Backport fix for DNS over TLS not working

• DNS over TLS not working #5550

[mdrahman-suse]

Validated on release-1.27 branch with commit 073d6d4

Environment/Config

Ubuntu 22.04

$ cat /etc/rancher/rke2/config.yaml
write-kubeconfig-mode: 644
token: summerheat
node-name: server1
debug: true

Testing steps

Thanks to Manuel for the detailed steps
Followed the steps mentioned here: #5550 (comment)

Replication:

$ rke2 -v
rke2 version v1.27.11+rke2r1 (6665618680112568f79b1f5992aecf4655e3cf8b)
go version go1.21.7 X:boringcrypto

• Before editing coredns configmap

kube-system   pod/rke2-coredns-rke2-coredns-94745d-bl295                 1/1     Running

$ k logs -n kube-system pod/rke2-coredns-rke2-coredns-94745d-bl295
.:53
[INFO] plugin/reload: Running configuration SHA512 = c18591e7950724fe7f26bd172b7e98b6d72581b4a8fc4e5fc4cfd08229eea58f4ad043c9fd3dbd1110a11499c4aa3164cdd63ca0dd5ee59651d61756c4f671b7
CoreDNS-1.11.1
linux/amd64, go1.20.7 X:boringcrypto, ae2bbc29

• After editing coredns configmap and restarting coredns pod

kube-system   pod/rke2-coredns-rke2-coredns-69d858bc84-hszhn             1/1     Running

$ k logs -n kube-system pod/rke2-coredns-rke2-coredns-69d858bc84-hszhn
.:53
[INFO] plugin/reload: Running configuration SHA512 = 6ac0e5f74fb935b974b697f3549b0d8c483c18fbf722f98f4c788c5b6059e96c4f9b7ed2f647a941bad8c7906563fefda900099aa7fcaa8756ea1fbcb6a8af5c
CoreDNS-1.11.1
linux/amd64, go1.20.7 X:boringcrypto, ae2bbc29
[ERROR] plugin/errors: 2 6447489085711979541.2734543790467838956. HINFO: tls: failed to verify certificate: x509: certificate signed by unknown authority

Validation:

$ rke2 -v
rke2 version v1.27.11+dev.073d6d40 (073d6d40edbd50fd62a1a021b86b65900a44fb3f)
go version go1.21.7 X:boringcrypto

• Before editing coredns configmap

kube-system   pod/rke2-coredns-rke2-coredns-864fbd7785-bzrtj              1/1     Running

$ k logs -n kube-system  pod/rke2-coredns-rke2-coredns-864fbd7785-bzrtj
.:53
[INFO] plugin/reload: Running configuration SHA512 = c18591e7950724fe7f26bd172b7e98b6d72581b4a8fc4e5fc4cfd08229eea58f4ad043c9fd3dbd1110a11499c4aa3164cdd63ca0dd5ee59651d61756c4f671b7
CoreDNS-1.11.1
linux/amd64, go1.20.14 X:boringcrypto, ae2bbc29

• After editing coredns configmap and restarting coredns pod

kube-system   pod/rke2-coredns-rke2-coredns-545d86c9c8-mn4st              1/1     Running

$ k logs -n kube-system pod/rke2-coredns-rke2-coredns-545d86c9c8-mn4st
.:53
[INFO] plugin/reload: Running configuration SHA512 = 6ac0e5f74fb935b974b697f3549b0d8c483c18fbf722f98f4c788c5b6059e96c4f9b7ed2f647a941bad8c7906563fefda900099aa7fcaa8756ea1fbcb6a8af5c
CoreDNS-1.11.1
linux/amd64, go1.20.14 X:boringcrypto, ae2bbc29