[Release-1.27] - Panic when etcd-snapshot-dir does not exist

[brandond]

Backport fix for Panic when etcd-snapshot-dir does not exist

• Panic when etcd-snapshot-dir does not exist #9316

[brandond]

Moving this out to the next release to fix the prune subcommand.

[aganesh-suse]

Validated on release-1.27 branch with commit 1268cf0

Environment Details

Infrastructure

• Cloud
• Hosted

Node(s) CPU architecture, OS, and Version:

$ cat /etc/os-release
PRETTY_NAME="Ubuntu 22.04.2 LTS"

$ uname -m
x86_64

Cluster Configuration:

HA: 3 server/ 1 agent

Config.yaml:

token: xxxx
cluster-init: true
write-kubeconfig-mode: "0644"
node-external-ip: 1.1.1.1
node-label:
- k3s-upgrade=server

etcd-snapshot-retention: 2
etcd-snapshot-schedule-cron: "* * * * *"
etcd-s3: true
etcd-s3-access-key: xxxx
etcd-s3-secret-key: xxxx
etcd-s3-bucket: xxxx
etcd-s3-folder: xxxx
etcd-s3-region: xxxx

etcd-snapshot-dir: '/does/not/exist'

debug: true

Testing Steps

1. Copy config.yaml

$ sudo mkdir -p /etc/rancher/k3s && sudo cp config.yaml /etc/rancher/k3s

2. Install k3s

curl -sfL https://get.k3s.io | sudo INSTALL_K3S_COMMIT='1268cf0a02e509071ab8843129b8561aed3a1ffd' sh -s - server

3. Verify Cluster Status:

kubectl get nodes -o wide
kubectl get pods -A

4. Verify etcd-snapshot operations: save, prune, list, delete

sudo /usr/local/bin/k3s etcd-snapshot list
sudo /usr/local/bin/k3s etcd-snapshot save
sudo /usr/local/bin/k3s etcd-snapshot prune
sudo /usr/local/bin/k3s etcd-snapshot delete <snapshot>

Validation Results:

• k3s version used for validation:

k3s version v1.27.11+k3s-1268cf0a (1268cf0a)
go version go1.21.7

Save:

 $ sudo /usr/local/bin/k3s etcd-snapshot save 
time="2024-03-11T18:42:04Z" level=warning msg="Unknown flag --token found in config.yaml, skipping\n"
time="2024-03-11T18:42:04Z" level=warning msg="Unknown flag --etcd-snapshot-schedule-cron found in config.yaml, skipping\n"
time="2024-03-11T18:42:04Z" level=warning msg="Unknown flag --cluster-init found in config.yaml, skipping\n"
time="2024-03-11T18:42:04Z" level=warning msg="Unknown flag --write-kubeconfig-mode found in config.yaml, skipping\n"
time="2024-03-11T18:42:04Z" level=warning msg="Unknown flag --node-external-ip found in config.yaml, skipping\n"
time="2024-03-11T18:42:04Z" level=warning msg="Unknown flag --node-label found in config.yaml, skipping\n"
time="2024-03-11T18:42:04Z" level=debug msg="Attempting to retrieve extra metadata from k3s-etcd-snapshot-extra-metadata ConfigMap"
time="2024-03-11T18:42:04Z" level=debug msg="Error encountered attempting to retrieve extra metadata from k3s-etcd-snapshot-extra-metadata ConfigMap, error: configmaps \"k3s-etcd-snapshot-extra-metadata\" not found"
time="2024-03-11T18:42:04Z" level=fatal msg="failed to get etcd-snapshot-dir: stat /does/not/exist: no such file or directory"

Prune:

 $ sudo /usr/local/bin/k3s etcd-snapshot prune
time="2024-03-11T18:43:11Z" level=warning msg="Unknown flag --token found in config.yaml, skipping\n"
time="2024-03-11T18:43:11Z" level=warning msg="Unknown flag --etcd-snapshot-schedule-cron found in config.yaml, skipping\n"
time="2024-03-11T18:43:11Z" level=warning msg="Unknown flag --cluster-init found in config.yaml, skipping\n"
time="2024-03-11T18:43:11Z" level=warning msg="Unknown flag --write-kubeconfig-mode found in config.yaml, skipping\n"
time="2024-03-11T18:43:11Z" level=warning msg="Unknown flag --node-external-ip found in config.yaml, skipping\n"
time="2024-03-11T18:43:11Z" level=warning msg="Unknown flag --node-label found in config.yaml, skipping\n"
time="2024-03-11T18:43:11Z" level=fatal msg="failed to get etcd-snapshot-dir: stat /does/not/exist: no such file or directory"

List:

$ sudo k3s etcd-snapshot list
WARN[0000] Unknown flag --token found in config.yaml, skipping
WARN[0000] Unknown flag --server found in config.yaml, skipping
WARN[0000] Unknown flag --write-kubeconfig-mode found in config.yaml, skipping
WARN[0000] Unknown flag --node-external-ip found in config.yaml, skipping
WARN[0000] Unknown flag --node-label found in config.yaml, skipping
FATA[0000] failed to get etcd-snapshot-dir: stat /does/not/exist: no such file or directory

Delete:

$ sudo k3s etcd-snapshot delete on-demand-ip-x
WARN[0000] Unknown flag --token found in config.yaml, skipping
WARN[0000] Unknown flag --server found in config.yaml, skipping
WARN[0000] Unknown flag --write-kubeconfig-mode found in config.yaml, skipping
WARN[0000] Unknown flag --node-external-ip found in config.yaml, skipping
WARN[0000] Unknown flag --node-label found in config.yaml, skipping
FATA[0000] failed to get etcd-snapshot-dir: stat /does/not/exist: no such file or directory