Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

pubsubplus-ha chart PVC gets stuck on PreSync hook when we deploy it via ArgoCD #144

Open
nikhilbhadane10 opened this issue Apr 3, 2024 · 6 comments

Comments

@nikhilbhadane10
Copy link

nikhilbhadane10 commented Apr 3, 2024

Hi Team,
We are trying to deploy the pubsubplus-ha via ArgoCD, But while creating the PVC it get stuck on PVC PreSync hook.

Message: "waiting for completion of hook /PersistentVolumeClaim/data-solace-pubsubplus-internal-2"

We tried to deploy the helm chart on AWS (EKS) & Azure (AKS) via ArgoCD app, but same result.
As per your documentation[1] & [2], we should use the storageClass with WaitForFirstConsumer binding mode. But that not works when we try to deploy the helm-chart via ArgoCD apps.

[1] https://docs.solace.com/Cloud/Deployment-Considerations/installing-ps-cloud-k8s-eks-specific-req.htm#eks_storage_class
[2] https://docs.solace.com/Cloud/Deployment-Considerations/installing-ps-cloud-k8s-aks-specific-req.htm

Is there any way we can fix this issue by doing some change on helm-chart templates or Can we use EFS/Azure file StorageClass for PVC creation? Is it recommended for prod environments?

@bczoma
Copy link
Collaborator

bczoma commented Apr 5, 2024

Hi @nikhilbhadane10 , this hook is controlled by https://github.com/SolaceProducts/pubsubplus-kubernetes-helm-quickstart/blob/master/pubsubplus/templates/solacePVCForMonitor.yaml
You could try to remove it in your local chart. However, chances are there may be general issues with the storage class and the rest of the deployment still fails. For comparison, you could also try if it works without the ArgoCD apps.

@nikhilbhadane10
Copy link
Author

nikhilbhadane10 commented Apr 10, 2024

Hi @bczoma ,
Thanks for your inputs, I have tried pubsubplus-ha by removing the solacePVCForMonitor.yaml file locally it started the deployment of statefulset but somehow it is failing with healthchecks. I have also tried by commenting the hook annotation from PVC manifests but no luck.
Is there any alternatives we can try to deploy the pubsubplus-ha with ArgoCD deployment.

@nikhilbhadane10
Copy link
Author

Hi Team,
Any update on the above issue?

@TheeeEnding
Copy link

Hi,
I'm having the same issues with the ArgoCD deployment. Would be great if there were some news on this issue as it makes monitoring unavailable for productiion environments using Argo...
Thanks!

@TheeeEnding
Copy link

I found an easy workaround that might help until there is a solution for this issue. Just manually run a temp pod with the PVC claim so the PVC gets a first consumer and all other resources are deployed. Then you can just delete the pod.
The pod could look like this:

apiVersion: v1
kind: Pod
metadata:
  name: task-pv-pod
  namespace: solace
spec:
  volumes:
    - name: task-pv-storage
      persistentVolumeClaim:
        claimName: data-solace-pubsubplus-2
  containers:
    - name: task-pv-container
      image: nginx
      volumeMounts:
        - mountPath: "/usr/share/nginx/html"
          name: task-pv-storage

@nikhilbhadane10
Copy link
Author

nikhilbhadane10 commented May 9, 2024

Readiness probes gets failed after some time. ArgoCD app goes into Progressing state. It seems same issue when we tried to delete solacePVCForMonitor.yaml from helm-chart & deploy it without PVC file via ArgoCD.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

3 participants