patch: refactor of entire code base

* updated security scanner * moved back to chart approach * helm chart and release process in this repo * entirely reworked how helm charts are deployed * completely rewritten tests
drewbernetes · Jan 3, 2024 · 56770a7 · 56770a7
1 parent 7fb47ec
commit 56770a7
Show file tree

Hide file tree

Showing 117 changed files with 2,670 additions and 4,925 deletions.
diff --git a/.github/workflows/tag.yml b/.github/workflows/tag.yml
@@ -47,9 +47,11 @@ jobs:
       - uses: actions/checkout@v4
         with:
           fetch-depth: 0
+
       - uses: actions/setup-go@v3
         with:
           go-version: '1.19.2'
+
       - name: Get OS and arch info
         run: |
           GOARCH=amd64
@@ -58,12 +60,15 @@ jobs:
           echo "BINARY_NAME=$BINARY_NAME" >> $GITHUB_ENV
           echo "GOOS=$GOOS" >> $GITHUB_ENV
           echo "GOARCH=$GOARCH" >> $GITHUB_ENV
+
       - name: Build
         run: |
           CGO_ENABLED=0 go build -o "$BINARY_NAME" cmd/dogkat/main.go
+
       - name: Release Notes
         run: |
           git log $(git describe HEAD~ --tags --abbrev=0)..HEAD --pretty='format:* %h %s - %an <%ae>' --no-merges >> ".github/RELEASE-TEMPLATE.md"
+
       - name: Release with Notes
         uses: softprops/action-gh-release@v1
         with:
@@ -84,4 +89,4 @@ jobs:
       - name: Run chart-releaser
         uses: helm/[email protected]
         env:
-          CR_TOKEN: '${{ secrets.GITHUB_TOKEN  }}'
+          CR_TOKEN: "${{ secrets.GITHUB_TOKEN }}"
diff --git a/.gitignore b/.gitignore
@@ -2,3 +2,4 @@
 .editorconfig
 .gitattributes
 bin
+dogkat.yaml
diff --git a/README.md b/README.md
@@ -61,3 +61,19 @@ Test GPU
 ```shell
 e2e-test validate gpu --number-of-gpus 1
 ```
+
+
+## Using The Chart
+
+```shell
+helm install oci:///charts/dogkat --values values.yaml
+```
+
+## Updating the Chart
+Make sure you've run and resolved any issues using the following as failures of these will cause the pipeline/actions to fail.
+```
+yamllint charts/dogkat/Chart.yaml  --config-file .github/ct.yaml
+yamllint charts/dogkat/values.yaml  --config-file .github/ct.yaml
+
+helm-docs
+```
diff --git a/charts/dogkat/Chart.yaml b/charts/dogkat/Chart.yaml
@@ -2,8 +2,8 @@
 
 apiVersion: v2
 name: dogkat
-version: 0.0.0
-appVersion: 0.0.0
+version: 0.1.0-beta.1
+appVersion: 0.1.0
 type: application
 maintainers:
   - name: drew-viles
@@ -19,61 +19,6 @@ keywords:
   - ingress
   - storage
 sources:
-  - https://github.com/eschercloudai/argocd-controller/tree/main/charts/dogkat
+  - https://github.com/eschercloudai/dogkat/tree/main/charts/dogkat
 description: |
-  # DogKat End-2-End testing
-
-  This setup will deploy a multi-replica deployment with a
-  web frontend and a database backend.
-  They are in reality not linked and do not have a valid application
-  but the test is to ensure we can spin things up
-  using affinity and ant-affinity, volumes and more.
-
-  ## Features
-  * Affinity/anti-affinity to attempt to evenly split workloads
-  * Multiple replicas of nginx and Postgresql
-  * Volume creation and mounting in Postgresql container
-  * Configmap mounting for Nginx Index.php for querying the Postgres sts
-  - confirms cluster dns works
-  * SQL seeding for populating DB on Post start lifecycle
-  - confirms cluster dns works
-  * Public and Private Ingress
-  * PDB to ensure pods stay online at all times
-  * Scripts to deploy, delete and test
-
-  # Tests
-  ## Ingress Testing
-
-  This workload will spin up an example deployment with an ingress
-  and certificate combo to confirm that ingress is working with TLS.
-
-  ## Affinity/Anti-Affinity Testing
-
-  This workload will spin up an example deployment to test affinity
-  and anti-affinity.
-  An Nginx and Postgres workload will be added to the cluster.
-
-  The Nginx pods should sit on different nodes to one another as
-  should the Postgres pods.
-  However, the Nginx pods should share a node with the Postgres and vice versa.
-
-  Once deployed, confirm that the pods are arranged as such onn each node.
-
-  *** Node X is just a random node in the EKS cluster, the pods are not
-  assigned to the nodes in any sort of order due to affinity settings ***
-
-  | **Node A**  | **Node B**    | **Node C**  |
-  |:------------|:--------------|:------------|
-  | `Nginx`     | `Nginx`       | `Nginx`     |
-  | `Postgres`  | `Postgres`    | `Postgres`  |
-
-  ## Storage Testing
-  The storage testing simply applies a PVC to the nginx pod
-  allowing for storage class testing.
-
-  ## PDB Testing
-  The PDB ensures a pod is always online to ensure no downtime during upgrades
-
-  ## Automated Testing:
-  Check out [E2E Tester](https://github.com/drew-viles/k8s-e2e-tester)
-  for an automated tester which will fire tests against this chart.
+  End-2-End testing for GPUs and some core resources
diff --git a/charts/dogkat/LICENSE.md b/charts/dogkat/LICENSE.md
@@ -1,4 +1,4 @@
-Copyright 2023 EscherCloud.
+Copyright 2024 EscherCloud.
 Licensed under the Apache License, Version 2.0 (the "License");
 you may not use this file except in compliance with the License.
 You may obtain a copy of the License at

diff --git a/charts/dogkat/README.md b/charts/dogkat/README.md
@@ -1,71 +1,17 @@
 # dogkat
 
-![Version: 0.0.0](https://img.shields.io/badge/Version-0.0.0-informational?style=flat-square) ![Type: application](https://img.shields.io/badge/Type-application-informational?style=flat-square) ![AppVersion: 0.0.0](https://img.shields.io/badge/AppVersion-0.0.0-informational?style=flat-square)
+![Version: 0.1.0-beta.1](https://img.shields.io/badge/Version-0.1.0--beta.1-informational?style=flat-square) ![Type: application](https://img.shields.io/badge/Type-application-informational?style=flat-square) ![AppVersion: 0.1.0](https://img.shields.io/badge/AppVersion-0.1.0-informational?style=flat-square)
 
-# DogKat End-2-End testing
-
-This setup will deploy a multi-replica deployment with a
-web frontend and a database backend.
-They are in reality not linked and do not have a valid application
-but the test is to ensure we can spin things up
-using affinity and ant-affinity, volumes and more.
-
-## Features
-* Affinity/anti-affinity to attempt to evenly split workloads
-* Multiple replicas of nginx and Postgresql
-* Volume creation and mounting in Postgresql container
-* Configmap mounting for Nginx Index.php for querying the Postgres sts
-- confirms cluster dns works
-* SQL seeding for populating DB on Post start lifecycle
-- confirms cluster dns works
-* Public and Private Ingress
-* PDB to ensure pods stay online at all times
-* Scripts to deploy, delete and test
-
-# Tests
-## Ingress Testing
-
-This workload will spin up an example deployment with an ingress
-and certificate combo to confirm that ingress is working with TLS.
-
-## Affinity/Anti-Affinity Testing
-
-This workload will spin up an example deployment to test affinity
-and anti-affinity.
-An Nginx and Postgres workload will be added to the cluster.
-
-The Nginx pods should sit on different nodes to one another as
-should the Postgres pods.
-However, the Nginx pods should share a node with the Postgres and vice versa.
-
-Once deployed, confirm that the pods are arranged as such onn each node.
-
-*** Node X is just a random node in the EKS cluster, the pods are not
-assigned to the nodes in any sort of order due to affinity settings ***
-
-| **Node A**  | **Node B**    | **Node C**  |
-|:------------|:--------------|:------------|
-| `Nginx`     | `Nginx`       | `Nginx`     |
-| `Postgres`  | `Postgres`    | `Postgres`  |
-
-## Storage Testing
-The storage testing simply applies a PVC to the nginx pod
-allowing for storage class testing.
-
-## PDB Testing
-The PDB ensures a pod is always online to ensure no downtime during upgrades
-
-## Automated Testing:
-Check out [E2E Tester](https://github.com/drew-viles/k8s-e2e-tester)
-for an automated tester which will fire tests against this chart.
+End-2-End testing for GPUs and some core resources
 
 ## Installation
 
 ```shell
-helm install oci://harbor.infra.nl1.eschercloud.dev/charts/dogkat --values values.yaml
+helm install https://eschercloudai.github.io/dogkat/dogkat --values values.yaml
 ```
 
 ## Updating the Chart
+
 Make sure you've run and resolved any issues using the following as failures of these will cause the pipeline/actions to fail.
 ```
 yamllint charts/dogkat/Chart.yaml  --config-file .github/ct.yaml
@@ -82,6 +28,7 @@ The following table lists the configurable parameters of the chart and the defau
 
 | Key | Type | Default | Description |
 |-----|------|---------|-------------|
+| core.enabled | bool | `false` |  |
 | core.nginx.image.repo | string | `"nginx"` | The repo to be used |
 | core.nginx.image.tag | string | `"1.25-alpine"` | The tag to be used |
 | core.nginx.resources | object | `{}` |  |
@@ -90,18 +37,18 @@ The following table lists the configurable parameters of the chart and the defau
 | core.php.image.tag | string | `"v1.1.0"` | The tag to be used |
 | core.postgres.image.repo | string | `"postgres"` | The repo to be used |
 | core.postgres.image.tag | string | `"16-alpine"` | The tag to be used |
-| core.postgres.statefulset.persistentData.enabled | bool | `true` |  |
-| core.postgres.statefulset.persistentData.storageClassName | string | `"cinder"` |  |
-| gpu.enabled | bool | `true` |  |
+| core.postgres.statefulSet.persistentData.enabled | bool | `true` |  |
+| core.postgres.statefulSet.persistentData.storageClassName | string | `"cinder"` |  |
+| gpu.enabled | bool | `false` |  |
 | gpu.image.repo | string | `"nvidia/samples"` | The repo to be used |
 | gpu.image.tag | string | `"vectoradd-cuda11.2.1"` | The tag to be used |
 | gpu.numberOfGPUs | int | `1` |  |
 | gpu.resources | object | `{}` |  |
 | ingress.annotations | object | `{}` |  |
 | ingress.className | string | `"nginx"` |  |
-| ingress.enabled | bool | `true` |  |
+| ingress.enabled | bool | `false` |  |
 | ingress.host | string | `"test.example.uk"` |  |
 | ingress.tls[0].hosts[0] | string | `"test.example.uk"` |  |
 | ingress.tls[0].secretName | string | `"test-secret"` |  |
-| monitoring.grafana.dashboards | bool | `true` |  |
-| monitoring.serviceMonitor.enabled | bool | `true` |  |
+| monitoring.grafana.dashboards | bool | `false` |  |
+| monitoring.serviceMonitor.enabled | bool | `false` |  |
diff --git a/charts/dogkat/README.md.gotmpl b/charts/dogkat/README.md.gotmpl
@@ -10,10 +10,11 @@
 ## Installation
 
 ```shell
-helm install oci://harbor.infra.nl1.eschercloud.dev/charts/{{ template "chart.name" . }} --values values.yaml
+helm install https://eschercloudai.github.io/dogkat/{{ template "chart.name" . }} --values values.yaml
 ```
 
 ## Updating the Chart
+
 Make sure you've run and resolved any issues using the following as failures of these will cause the pipeline/actions to fail.
 ```
 yamllint charts/{{ template "chart.name" . }}/Chart.yaml  --config-file .github/ct.yaml

diff --git a/charts/dogkat/templates/gpu-pod.yaml b/charts/dogkat/templates/gpu-pod.yaml
@@ -1,10 +1,10 @@
 {{- if .Values.gpu.enabled -}}
 ---
 
-apiVersion: apps/v1
+apiVersion: v1
 kind: Pod
 metadata:
-  name: nginx-e2e
+  name: gpu-test
   namespace: {{ .Release.Namespace }}
   annotations:
     {{- include "e2e-testing.annotations" . | nindent 4 }}
@@ -13,12 +13,14 @@ metadata:
     {{- include "e2e-testing.labels" . | nindent 4 }}
     {{- include "e2e-testing.gpu.labels" . | nindent 4 }}
 spec:
+  restartPolicy: OnFailure
   containers:
-    - name: nginx
+    - name: nvidia-vectoradd
       image: {{.Values.gpu.image.repo}}:{{.Values.gpu.image.tag}}
       terminationMessagePath: /dev/termination-log
       terminationMessagePolicy: File
       imagePullPolicy: IfNotPresent
+
       {{- if .Values.gpu.resources }}
       resources:
         "nvidia.com/gpu": {{ .Values.gpu.numberOfGPUs }}

diff --git a/charts/dogkat/templates/nginx-configmap.yaml b/charts/dogkat/templates/nginx-configmap.yaml
@@ -1,3 +1,4 @@
+{{- if or .Values.core.enabled  .Values.ingress.enabled -}}
 ---
 apiVersion: v1
 kind: ConfigMap
@@ -47,3 +48,4 @@ data:
           stub_status on;
       }
     }
+{{- end }}
diff --git a/charts/dogkat/templates/nginx-deploy.yaml b/charts/dogkat/templates/nginx-deploy.yaml
@@ -1,3 +1,4 @@
+{{- if or .Values.core.enabled  .Values.ingress.enabled -}}
 ---
 
 apiVersion: apps/v1
@@ -132,3 +133,4 @@ spec:
                 path: default.conf
               - key: metrics
                 path: metrics.conf
+{{- end }}
diff --git a/charts/dogkat/templates/nginx-grafana-dashboard.yaml b/charts/dogkat/templates/nginx-grafana-dashboard.yaml
@@ -1,4 +1,4 @@
-{{- if .Values.monitoring.grafana.dashboards -}}
+{{- if and .Values.monitoring.grafana.dashboards (or .Values.core.enabled  .Values.ingress.enabled) -}}
 ---
 
 apiVersion: v1

diff --git a/charts/dogkat/templates/nginx-pdb.yaml b/charts/dogkat/templates/nginx-pdb.yaml
@@ -1,3 +1,4 @@
+{{- if or .Values.core.enabled  .Values.ingress.enabled -}}
 ---
 
 apiVersion: policy/v1
@@ -16,3 +17,4 @@ spec:
   selector:
     matchLabels:
       app: web-frontend
+{{- end }}
diff --git a/charts/dogkat/templates/nginx-servicemonitor.yaml b/charts/dogkat/templates/nginx-servicemonitor.yaml
@@ -1,4 +1,4 @@
-{{- if .Values.monitoring.serviceMonitor.enabled -}}
+{{- if and .Values.monitoring.serviceMonitor.enabled (or .Values.core.enabled  .Values.ingress.enabled) -}}
 ---
 apiVersion: monitoring.coreos.com/v1
 kind: ServiceMonitor

diff --git a/charts/dogkat/templates/nginx-svc-account.yaml b/charts/dogkat/templates/nginx-svc-account.yaml
@@ -1,3 +1,4 @@
+{{- if or .Values.core.enabled  .Values.ingress.enabled -}}
 ---
 
 apiVersion: v1
@@ -11,3 +12,4 @@ metadata:
   labels:
     {{- include "e2e-testing.labels" . | nindent 4 }}
     {{- include "e2e-testing.nginx.labels" . | nindent 4 }}
+{{- end }}
diff --git a/charts/dogkat/templates/nginx-svc.yaml b/charts/dogkat/templates/nginx-svc.yaml
@@ -1,3 +1,5 @@
+{{- if or .Values.core.enabled  .Values.ingress.enabled -}}
+---
 apiVersion: v1
 kind: Service
 metadata:
@@ -22,3 +24,4 @@ spec:
   selector:
     app: web-frontend
   type: ClusterIP
+{{- end }}
diff --git a/charts/dogkat/templates/sql-deploy.yaml b/charts/dogkat/templates/sql-deploy.yaml
@@ -1,3 +1,4 @@
+{{- if or .Values.core.enabled  .Values.ingress.enabled -}}
 ---
 apiVersion: apps/v1
 kind: StatefulSet
@@ -51,7 +52,7 @@ spec:
             - -c
             - rm -rf /var/lib/postgresql/data/*
             - rm -rf /var/lib/postgresql/data/.*
-          {{- if (eq true .Values.core.postgres.statefulset.persistentData.enabled)}}
+          {{- if (eq true .Values.core.postgres.statefulSet.persistentData.enabled)}}
           volumeMounts:
             - name: data
               mountPath: "/var/lib/postgresql/data"
@@ -84,7 +85,7 @@ spec:
           configMap:
             name: sql-init
             defaultMode: 0755
-      {{- if (eq false .Values.core.postgres.statefulset.persistentData.enabled)}}
+      {{- if (eq false .Values.core.postgres.statefulSet.persistentData.enabled)}}
         - name: data
           emptyDir: { }
       {{- else }}
@@ -93,9 +94,11 @@ spec:
         name: data
       spec:
         accessModes: [ "ReadWriteOnce" ]
-        storageClassName: {{.Values.core.postgres.statefulset.persistentData.storageClassName}}
+        storageClassName: {{.Values.core.postgres.statefulSet.persistentData.storageClassName }}
         volumeMode: Filesystem
         resources:
           requests:
             storage: 10Gi
+      {{- end }}
+
 {{- end }}
diff --git a/charts/dogkat/templates/sql-init-configmap.yaml b/charts/dogkat/templates/sql-init-configmap.yaml
@@ -1,3 +1,4 @@
+{{- if or .Values.core.enabled  .Values.ingress.enabled -}}
 ---
 apiVersion: v1
 kind: ConfigMap
@@ -24,3 +25,4 @@ data:
     	CREATE TABLE IF NOT EXISTS web (id SERIAL PRIMARY KEY, value VARCHAR(10) NOT NULL );
         INSERT INTO web (value) VALUES('ok');
     EOF
+{{- end }}