monitoring deployment revision

[x-Xymos]

• Install grafana/prometheus/loki using helm
• Installs a local-path-provisioner which dynamically provisions PersistentVolumes rather than having to create them manually along with their required mount paths
• monitoring services are accessible from the ingress at a local domain - strims.monitoring.local

[x-Xymos]

@slugalisk looking through the old files I noticed some pod constraints in there and to my understanding you intended for the core monitoring services to run on the master? I added that back in just in case

NAME                                                     STATUS    NODE         RESTARTS
loki-0                                                   Running   controller   0
loki-promtail-68z8v                                      Running   worker2      0
loki-promtail-lkvch                                      Running   worker6      0
loki-promtail-rcxjs                                      Running   controller   0
loki-promtail-thmgj                                      Running   worker4      0
loki-promtail-v2sxj                                      Running   worker3      0
loki-promtail-wh688                                      Running   worker5      0
loki-promtail-zq62n                                      Running   worker1      0
prometheus-operator-grafana-59f49859f5-qrr7z             Running   controller   0
prometheus-operator-kube-state-metrics-d6946f59b-z2djs   Running   controller   0
prometheus-operator-operator-7cd6b7d9d7-s9cng            Running   controller   0
prometheus-operator-prometheus-node-exporter-26zvx       Running   controller   0
prometheus-operator-prometheus-node-exporter-bnvcx       Running   worker2      0
prometheus-operator-prometheus-node-exporter-hnh96       Running   worker5      0
prometheus-operator-prometheus-node-exporter-j76xn       Running   worker3      0
prometheus-operator-prometheus-node-exporter-nkttp       Running   worker1      0
prometheus-operator-prometheus-node-exporter-v82qk       Running   worker4      0
prometheus-operator-prometheus-node-exporter-xjz5f       Running   worker6      0
prometheus-prometheus-operator-prometheus-0              Running   controller   0

[slugalisk]

@x-Xymos yeah, the master node is the only one with enough memory/storage rn to not fall over hosting these.

avoiding fragmenting infra setup by having everything in helm seems like a good idea but isn't github.com/helm/charts deprecated? i looked into this when i set up prometheus and it seemed like a year after the announcement no one cared enough about the prometheus chart to migrate it to its own repo for helm 3. maybe i misread the situation, do we think this is going to continue to be supported?

local persistent volumes are part of kuberentes core now. should we still be using the local-path-provisioner? it's a bit annoying that it launched without automated provisioning but again i'm wondering which solution is going to continue to be supported...

[x-Xymos]

@slugalisk I believe the migration has been done for prometheus prometheus-community/community#28 (comment) https://github.com/prometheus-community/helm-charts
It's a fairly popular project so I don't see it not being supported in the future.

Having automated local storage provisioning mimics the way a disk would be provisioned dynamically on the cloud without us taking special measures, it uses local persistent volumes and does the work of automatically creating the paths on the host.

Local Path Provisioner provides a way for the Kubernetes users to utilize the local storage in each node. Based on the user configuration, the Local Path Provisioner will create hostPath based persistent volume on the node automatically. It utilizes the features introduced by Kubernetes Local Persistent Volume feature, but make it a simpler solution than the built-in local volume feature in Kubernetes.

[slugalisk]

oh sweet, nothing like waiting to the last possible moment.

i didn't realize local-path was a wrapper around the core local-storage feature. thanks for clarifying.

lgtm 👍