Recommend democratic-csi for local storage instead of local-path-provisioner?

[justyns]

https://www.talos.dev/v1.9/kubernetes-guides/configuration/local-storage/ recommends using https://github.com/rancher/local-path-provisioner which works but has a caveat of not pinning workloads to specific machines.

E.g. if I deploy something with a local-path-provisioner pvc, kubernetes doesn't guarantee that workload will always be deployed to the same node. This basically makes the pvc ephemeral unless I'm missing something.

Instead, could/should https://github.com/democratic-csi/democratic-csi be recommended as an alternative? It's local-hostpath does cause workloads to always be deployed to the node where the local-hostpath volume was originally created.

[runningman84]

From my point of view it is pinning the workload to specific node:

apiVersion: v1
kind: PersistentVolume
metadata:
  annotations:
    local.path.provisioner/selected-node: worker1
    pv.kubernetes.io/provisioned-by: cluster.local/local-path-provisioner
  creationTimestamp: "2024-05-26T09:09:31Z"
  finalizers:
  - kubernetes.io/pv-protection
  name: pvc-edec79a4-ab61-477e-bf1d-b8bdb4300525
  resourceVersion: "156299"
  uid: 8a080682-71d3-4322-b88c-55587f417922
spec:
  accessModes:
  - ReadWriteOnce
  capacity:
    storage: 1Gi
  claimRef:
    apiVersion: v1
    kind: PersistentVolumeClaim
    name: alertmanager-prometheus-operator-kube-p-alertmanager-db-alertmanager-prometheus-operator-kube-p-alertmanager-0
    namespace: monitoring
    resourceVersion: "156167"
    uid: edec79a4-ab61-477e-bf1d-b8bdb4300525
  hostPath:
    path: /var/lib/local-path-provisioner/pvc-edec79a4-ab61-477e-bf1d-b8bdb4300525_monitoring_alertmanager-prometheus-operator-kube-p-alertmanager-db-alertmanager-prometheus-operator-kube-p-alertmanager-0
    type: DirectoryOrCreate
  nodeAffinity:
    required:
      nodeSelectorTerms:
      - matchExpressions:
        - key: kubernetes.io/hostname
          operator: In
          values:
          - worker1
  persistentVolumeReclaimPolicy: Delete
  storageClassName: local-path
  volumeMode: Filesystem
status:
  lastPhaseTransitionTime: "2024-05-26T09:09:31Z"
  phase: Bound

[linucksrox]

Yes, you can use democratic-csi to decouple the PVC from the node and run the pod anywhere in the cluster. I have done this along with a TrueNAS instance using NFS and iSCSI, although I couldn't quite get NVMe working. I wrote about this in detail: https://blog.dalydays.com/post/kubernetes-homelab-series-part-6-storage-with-democratic-csi/