[Release-1.28] - Segmentation Fault on running rke2 etcd-snapshot save

[brandond]

Backport fix for Segmentation Fault on running rke2 etcd-snapshot save

• Segmentation Fault on running rke2 etcd-snapshot save #4942

[brandond]

Moving out to the next release; save no longer crashes if the directory doesn't exist, but prune still does.

[aganesh-suse]

Validated on release-1.28 branch with commit 1a021e6

Environment Details

Infrastructure

• Cloud
• Hosted

Node(s) CPU architecture, OS, and Version:

$ cat /etc/os-release
PRETTY_NAME="Ubuntu 22.04.2 LTS"

$ uname -m
x86_64

Cluster Configuration:

HA : 3 server / 1 agent

Config.yaml:

token: xxxx
write-kubeconfig-mode: "0644"
node-external-ip: 1.1.1.1
debug: true

etcd-snapshot-retention: 5
etcd-snapshot-schedule-cron: "* * * * *"
etcd-s3: true
etcd-s3-access-key: xxxx
etcd-s3-secret-key: xxxx
etcd-s3-bucket: xxxx
etcd-s3-folder: xxxx
etcd-s3-region: xxxx

etcd-snapshot-dir: '/does/not/exist'

Testing Steps

1. Copy config.yaml

$ sudo mkdir -p /etc/rancher/rke2 && sudo cp config.yaml /etc/rancher/rke2

2. Install RKE2

curl -sfL https://get.rke2.io | sudo INSTALL_RKE2_COMMIT='1a021e6a4d5872bda9304a8702581be598ea77a4' INSTALL_RKE2_TYPE='server' INSTALL_RKE2_METHOD=tar sh -

3. Start the RKE2 service

$ sudo systemctl enable --now rke2-server
or 
$ sudo systemctl enable --now rke2-agent

4. Verify Cluster Status:

kubectl get nodes -o wide
kubectl get pods -A

5. Perform etcd-snapshot operations: list, save, prune, delete. They should exit gracefully.

Validation Results:

• rke2 version used for validation:

$ rke2 -v
rke2 version v1.28.7+dev.1a021e6a (1a021e6a4d5872bda9304a8702581be598ea77a4)
go version go1.21.7 X:boringcrypto

List:

$ sudo rke2 etcd-snapshot list
WARN[0000] Unknown flag --token found in config.yaml, skipping
WARN[0000] Unknown flag --server found in config.yaml, skipping
WARN[0000] Unknown flag --etcd-snapshot-schedule-cron found in config.yaml, skipping
WARN[0000] Unknown flag --write-kubeconfig-mode found in config.yaml, skipping
WARN[0000] Unknown flag --node-external-ip found in config.yaml, skipping
INFO[0000] Checking if S3 bucket sonobuoy-results exists
INFO[0000] S3 bucket sonobuoy-results exists
FATA[0000] failed to get etcd-snapshot-dir: stat /does/not/exist: no such file or directory

Save:

$ sudo rke2 etcd-snapshot save
WARN[0000] Unknown flag --token found in config.yaml, skipping
WARN[0000] Unknown flag --server found in config.yaml, skipping
WARN[0000] Unknown flag --etcd-snapshot-schedule-cron found in config.yaml, skipping
WARN[0000] Unknown flag --write-kubeconfig-mode found in config.yaml, skipping
WARN[0000] Unknown flag --node-external-ip found in config.yaml, skipping
DEBU[0000] Attempting to retrieve extra metadata from rke2-etcd-snapshot-extra-metadata ConfigMap
DEBU[0000] Error encountered attempting to retrieve extra metadata from rke2-etcd-snapshot-extra-metadata ConfigMap, error: configmaps "rke2-etcd-snapshot-extra-metadata" not found
FATA[0000] failed to get etcd-snapshot-dir: stat /does/not/exist: no such file or directory

Prune:

$ sudo rke2 etcd-snapshot prune
WARN[0000] Unknown flag --token found in config.yaml, skipping
WARN[0000] Unknown flag --server found in config.yaml, skipping
WARN[0000] Unknown flag --etcd-snapshot-schedule-cron found in config.yaml, skipping
WARN[0000] Unknown flag --write-kubeconfig-mode found in config.yaml, skipping
WARN[0000] Unknown flag --node-external-ip found in config.yaml, skipping
FATA[0000] failed to get etcd-snapshot-dir: stat /does/not/exist: no such file or directory

Delete:

$ sudo rke2 etcd-snapshot delete etcd-snapshot-x
WARN[0000] Unknown flag --token found in config.yaml, skipping
WARN[0000] Unknown flag --server found in config.yaml, skipping
WARN[0000] Unknown flag --etcd-snapshot-schedule-cron found in config.yaml, skipping
WARN[0000] Unknown flag --write-kubeconfig-mode found in config.yaml, skipping
WARN[0000] Unknown flag --node-external-ip found in config.yaml, skipping
FATA[0000] failed to get etcd-snapshot-dir: stat /does/not/exist: no such file or directory