Velero can’t send the snapshots of longhorn disks to vmware object storage

[burhan-b]

What steps did you take and what happened:

Deployed:
https://github.com/kubernetes-csi/external-snapshotter/blob/master/client/config/crd/kustomization.yaml
https://github.com/kubernetes-csi/external-snapshotter/blob/master/deploy/kubernetes/csi-snapshotter/kustomization.yaml
https://github.com/kubernetes-csi/external-snapshotter/blob/master/deploy/kubernetes/snapshot-controller/kustomization.yaml

Then installed helm chart:
helm repo add vmware-tanzu https://vmware-tanzu.github.io/helm-charts
helm install velero vmware-tanzu/velero --namespace velero-test -f values-5.2.0.yaml

I tried to test it with a 1 GB volume which is mounted to test-ss statefulset nginx app. But I’d like to take all volumes’ snapshots in mongodb namespace.

Status:
Velero sends metadata to vmware object storage but can’t send snapshots which I specified for a namespace in schedules.

What did you expect to happen:

1. Send all volumes’ snapshots in mongodb namespace

Next Goals

1. Take incremental backups everyday (metadata + snapshots)
2. Take full backups every week (metadata + snapshots)

The output of the following commands will help us better understand what's going on:
My values file and velero pod outputs (loglevel info and warning) are in https://gist.github.com/burhan-b/515c452c16cc07afd13086fb88b41123

Anything else you would like to add:
[Miscellaneous information that will assist in solving the issue.]

Environment:

• helm version (use helm version): 5.2.0
• helm chart version and app version (use helm list -n <YOUR NAMESPACE>): velero-test
• Kubernetes version (use kubectl version): 1.26.5
• Kubernetes installer & version:
• Cloud provider or hardware configuration: bare metal
• OS (e.g. from /etc/os-release): Ubuntu 20.04.4 LTS
• Longhorn: 1.5.1

[jenting]

Have you tried other backup target such as MinIO?

[burhan-b]

No, I've not tried MiniO. I've created a python script which uses longhorn api to trigger the backup&snapshot process. I've used velero to take and send metada.

[jenting]

I've created a python script which uses longhorn api to trigger the backup&snapshot process.

So, you aren't using Velero + Velero CSI plugin + Velero schedule CRs to take snapshot/restore the volume data?

[burhan-b]

Right. I couldn't use "take snapshot/restore the volume data" property with velero components.