Skip to content

Commit

Permalink
Merge pull request #17 from ccremer/initialize-znapzend
Browse files Browse the repository at this point in the history
[znapzend] Initialize znapzend with Backup plans
  • Loading branch information
ccremer authored May 4, 2020
2 parents fd85758 + f2f1c00 commit 305000a
Show file tree
Hide file tree
Showing 17 changed files with 508 additions and 62 deletions.
2 changes: 1 addition & 1 deletion znapzend/Chart.yaml
Original file line number Diff line number Diff line change
Expand Up @@ -14,7 +14,7 @@ description: Znapzend Helm Chart for automated ZFS snapshot & replication

# This is the chart version. This version number should be incremented each time you make changes
# to the chart and its templates, including the app version.
version: 0.4.2
version: 0.5.0

# This is the version number of the application being deployed. This version number should be
# incremented each time you make changes to the application.
Expand Down
25 changes: 14 additions & 11 deletions znapzend/README.md
Original file line number Diff line number Diff line change
Expand Up @@ -2,7 +2,7 @@ znapzend
========
Znapzend Helm Chart for automated ZFS snapshot & replication

Current chart version is `0.4.2`
Current chart version is `0.5.0`



Expand All @@ -13,24 +13,25 @@ Current chart version is `0.4.2`
| Key | Type | Default | Description |
|-----|------|---------|-------------|
| affinity | object | `{}` | |
| env | object | `{}` | A dict with KEY: VALUE pairs |
| env | object | `{}` | A dict with `KEY: VALUE` pairs |
| fullnameOverride | string | `""` | |
| host.zfsDevice | string | `"/dev/zfs"` | The device on the host which is used by the 'zfs' binary within the container |
| host.zfsDevice | string | `"/dev/zfs"` | The device on the host which is used by the `zfs` binary within the container |
| image.pullPolicy | string | `"IfNotPresent"` | |
| image.registry | string | `"docker.io"` | Znapzend image registry |
| image.repository | string | `"oetiker/znapzend"` | Znapzend image repository |
| image.tag | string | `"v0.20.0"` | Znapzend image tag (version) |
| imagePullSecrets | list | `[]` | List of image pull secrets if you use a privately hosted image |
| metrics.enabled | bool | `true` | Enable the znapzend metrics exporter for Prometheus |
| metrics.env | object | `{}` | A dict with KEY: VALUE pairs as environment variables for the exporter |
| metrics.enabled | bool | `true` | Enable the metrics exporter for Prometheus |
| metrics.env | object | `{}` | A dict with `KEY: VALUE` pairs as environment variables for the exporter |
| metrics.image.pullPolicy | string | `"IfNotPresent"` | |
| metrics.image.repository | string | `"docker.io/ccremer/znapzend-exporter"` | Exporter image repository |
| metrics.image.tag | string | `"v0.2.2"` | Exporter image tag |
| metrics.ingress.annotations | object | `{}` | |
| metrics.ingress.enabled | bool | `false` | Useful if your Prometheus is outside of the cluster |
| metrics.ingress.hosts | list | `[{"host":null,"paths":[]}]` | See Kubernetes Docs for a guide to setup TLS on Ingress |
| metrics.ingress.hosts | list | `[]` | See Kubernetes Docs for a guide to setup TLS on Ingress |
| metrics.ingress.tls | list | `[]` | |
| metrics.jobs.register | list | `[]` | String list of datasets that should be registered right at startup |
| metrics.port | int | `8080` | Container port to bind |
| metrics.resources.limits.memory | string | `"40Mi"` | |
| metrics.resources.requests.cpu | string | `"20m"` | |
| metrics.resources.requests.memory | string | `"20Mi"` | |
Expand All @@ -48,11 +49,13 @@ Current chart version is `0.4.2`
| securityContext | object | `{"allowPrivilegeEscalation":true,"privileged":true}` | The current image requires to run privileged in order to access ZFS |
| serviceAccount.annotations | object | `{}` | Annotations to add to the service account |
| serviceAccount.create | bool | `true` | Specifies whether a service account should be created |
| serviceAccount.name | string | `nil` | If not set and create is true, a name is generated using the fullname template |
| ssh.config | string | `nil` | ssh_config(5)-compatible file content to configure SSH options when connecting |
| ssh.externalSecretName | string | `nil` | Set this value if you provide your own secret with SSH config |
| ssh.identities | object | `{}` | Provide a private key for each SSH identity, see values.yaml for an example |
| ssh.knownHosts | string | `nil` | List of {host, pubKey} dicts where the public key of each host is configured |
| serviceAccount.name | string | `""` | If not set and create is true, a name is generated using the fullname template |
| ssh.config | string | `""` | `ssh_config(5)`-compatible file content to configure SSH options when connecting |
| ssh.externalSecretName | string | `""` | Set this value if you provide your own secret with SSH config |
| ssh.identities | object | `{}` | Provide a private key for each SSH identity, see [values.yaml](./values.yaml) for an example |
| ssh.knownHosts | list | `[]` | List of `{host, pubKey}` dicts where the public key of each host is configured |
| ssh.path | string | `"/root/.ssh"` | Path where your SSH config and identities get mounted in the container |
| tolerations | list | `[]` | |
| znapzend.args | list | `["znapzend","--logto=/dev/stdout","--autoCreation"]` | List of command arguments |
| znapzend.backupPlans | list | `[]` | List of backup plans to create/ensure on startup, see [values.yaml](./values.yaml) for an example |
| znapzend.reloadPlans | bool | `true` | Whether znapzend should reload the `znapzend.backupPlans` after modifying them. Creates additional RBAC roles for the `serviceAccount.name` |
30 changes: 30 additions & 0 deletions znapzend/templates/_helpers.tpl
Original file line number Diff line number Diff line change
Expand Up @@ -61,3 +61,33 @@ Create the name of the service account to use
{{ default "default" .Values.serviceAccount.name }}
{{- end -}}
{{- end -}}

{{/*
Define SelfReset
*/}}
{{- define "metrics.selfReset" -}}
SelfResetAfter={{ default "1h" .selfResetAfter }}
{{- end -}}

{{/*
Common Volumes
*/}}
{{- define "znapzend.volumes" -}}
- name: zfs
hostPath:
path: "{{ .Values.host.zfsDevice }}"
type: CharDevice
{{ $secretName := include "znapzend.fullname" . }}
{{- with .Values.ssh -}}
{{- if or .identities .externalSecretName }}
- name: ssh
secret:
{{- if .externalSecretName }}
secretName: {{ .externalSecretName }}
{{- else }}
secretName: {{ $secretName }}
{{- end }}
defaultMode: 0600
{{- end }}
{{- end }}
{{- end }}
32 changes: 32 additions & 0 deletions znapzend/templates/configmap.yaml
Original file line number Diff line number Diff line change
@@ -0,0 +1,32 @@
{{- if .Values.znapzend.backupPlans -}}
apiVersion: v1
kind: ConfigMap
metadata:
name: {{ include "znapzend.fullname" . }}-backup-plans
labels:
{{- include "znapzend.labels" . | nindent 4 }}
data:
{{- range $index, $plan := .Values.znapzend.backupPlans }}
plan{{ $index }}: {{ if $plan.custom }}{{ toYaml $plan.custom | indent 2 }}{{ else }}|
enabled={{ default "on" $plan.plan.enabled }}
src_plan={{ $plan.plan.source.retention }}
{{- range $index, $value := $plan.plan.targets }}
dst_{{ $index }}={{ $value.host }}:{{ $value.dataset }}
dst_{{ $index}}_plan={{ $value.retention }}
{{- if $.Values.metrics.enabled }}
dst_{{ $index }}_precmd=/usr/bin/curl -sS localhost:{{ $.Values.metrics.port }}/presend/{{ $plan.dataset }}
dst_{{ $index }}_pstcmd=/usr/bin/curl -sS localhost:{{ $.Values.metrics.port }}/postsend/{{ $plan.dataset }}?{{ include "metrics.selfReset" $plan.plan }}
{{- end }}
{{- end }}
{{- if $.Values.metrics.enabled }}
pre_znap_cmd=/usr/bin/curl -sS localhost:{{ $.Values.metrics.port }}/presnap/{{ $plan.dataset }}
post_znap_cmd=/usr/bin/curl -sS localhost:{{ $.Values.metrics.port }}/postsnap/{{ $plan.dataset }}{{- if not $plan.plan.targets }}?{{ include "metrics.selfReset" $plan.plan }}{{- end }}
{{- end }}
recursive={{- if $plan.plan.source.recursive }}on{{ else }}off{{- end }}
tsformat={{ default "%Y-%m-%d-%H%M%S" $plan.plan.tsFormat }}
mbuffer_size={{ default "1G" $plan.plan.mbufferSize }}
mbuffer={{ default "off" $plan.plan.mbuffer }}
zend_delay={{ default 0 $plan.plan.delaySeconds }}
{{- end }}
{{- end }}
{{- end }}
78 changes: 78 additions & 0 deletions znapzend/templates/job.yaml
Original file line number Diff line number Diff line change
@@ -0,0 +1,78 @@
{{- if .Values.znapzend.backupPlans -}}
apiVersion: batch/v1
kind: Job
metadata:
name: {{ include "znapzend.fullname" . }}-backup-plans
annotations:
helm.sh/hook: post-install,post-upgrade
spec:
backoffLimit: 2
ttlSecondsAfterFinished: 172800 # 2 days
template:
spec:
serviceAccountName: {{ include "znapzend.serviceAccountName" . }}
restartPolicy: OnFailure
containers:
- name: znapzendzetup
securityContext:
{{- toYaml .Values.securityContext | nindent 12 }}
image: "{{ .Values.image.registry }}/{{ .Values.image.repository }}:{{ .Values.image.tag }}"
imagePullPolicy: {{ .Values.image.pullPolicy }}
{{- with .Values.env }}
env:
{{- range $key, $value := . }}
- name: {{ $key }}
value: {{ $value }}
{{- end }}
{{- end }}
args:
- |
set -e &&
{{- range $index, $plan := .Values.znapzend.backupPlans }}
znapzendzetup import --write {{ $plan.dataset }} /etc/znapzend/plan{{ $index }} &&
{{- end }}
echo "$?" > /tmp/znapzend/success
volumeMounts:
- name: zfs
mountPath: /dev/zfs
{{- if or .Values.ssh.identities .Values.ssh.externalSecretName }}
- name: ssh
mountPath: {{ .Values.ssh.path }}
{{- end }}
- name: backup-plans
mountPath: /etc/znapzend
- name: success
mountPath: /tmp/znapzend
{{- if .Values.znapzend.reloadPlans }}
- name: reload-znapzend
image: "docker.io/bitnami/kubectl:1.17"
imagePullPolicy: {{ .Values.image.pullPolicy }}
command:
- /bin/bash
- -c
args:
- |
set -e
echo "waiting until backup plans are imported..."
until [ -f /tmp/znapzend/success ]; do
printf . && sleep 2
done
if [ $(cat /tmp/znapzend/success) == "0" ]; then
pod=$(kubectl get pods -o name -l app.kubernetes.io/name={{ include "znapzend.name" . }} | head -n 1)
kubectl exec -c znapzend "$pod" -- pkill -HUP znapzend
else
echo "Import failed"
exit 1
fi
volumeMounts:
- name: success
mountPath: /tmp/znapzend
{{- end }}
volumes:
{{- include "znapzend.volumes" . | nindent 8 }}
- name: backup-plans
configMap:
name: {{ include "znapzend.fullname" . }}-backup-plans
- name: success
emptyDir: {}
{{- end }}
31 changes: 31 additions & 0 deletions znapzend/templates/rbac.yaml
Original file line number Diff line number Diff line change
@@ -0,0 +1,31 @@
{{- if and .Values.znapzend.backupPlans .Values.znapzend.reloadPlans -}}
---
apiVersion: rbac.authorization.k8s.io/v1
kind: Role
metadata:
name: {{ include "znapzend.fullname" . }}-pod-exec-role
labels:
{{- include "znapzend.labels" . | nindent 4 }}
rules:
- apiGroups: [""]
resources: ["pods"]
verbs: ["get", "list"]
- apiGroups: [""]
resources: ["pods/exec"]
verbs: ["create"]
---
apiVersion: rbac.authorization.k8s.io/v1
kind: RoleBinding
metadata:
name: {{ include "znapzend.fullname" . }}-pod-exec-binding
labels:
{{- include "znapzend.labels" . | nindent 4 }}
roleRef:
apiGroup: rbac.authorization.k8s.io
kind: Role
name: {{ include "znapzend.fullname" . }}-pod-exec-role
subjects:
- kind: ServiceAccount
name: {{ include "znapzend.serviceAccountName" . }}
namespace: {{ .Release.Namespace }}
{{- end }}
22 changes: 4 additions & 18 deletions znapzend/templates/statefulset.yaml
Original file line number Diff line number Diff line change
Expand Up @@ -61,7 +61,9 @@ spec:
imagePullPolicy: {{ .image.pullPolicy }}
ports:
- name: http
containerPort: 8080
containerPort: {{ .port }}
args:
- --bindAddr=:{{ .port }}
env:
{{- if .jobs.register }}
- name: JOBS_REGISTER
Expand Down Expand Up @@ -102,20 +104,4 @@ spec:
{{- toYaml . | nindent 8 }}
{{- end }}
volumes:
- name: zfs
hostPath:
path: "{{ .Values.host.zfsDevice }}"
type: CharDevice
{{ $secretName := include "znapzend.fullname" . }}
{{- with .Values.ssh }}
{{- if or .identities .externalSecretName }}
- name: ssh
secret:
{{- if .externalSecretName }}
secretName: {{ .externalSecretName }}
{{- else }}
secretName: {{ $secretName }}
{{- end }}
defaultMode: 0600
{{- end }}
{{- end }}
{{- include "znapzend.volumes" . | nindent 8 -}}
Loading

0 comments on commit 305000a

Please sign in to comment.