chore: Update helm chart to 7.0.3

.github/workflows/migrate-s3.yaml

@@ -69,10 +69,12 @@ jobs:
           # remove all spaces and put one comma between prefixes
           cleaned_prefixes=$(echo $prefixes | tr -s '[:space:]' ',' | sed 's/[,/]*$//')
 
-          # remove `contentstore.deleted/` from cleaned_prefixes with comma if in list
-          cleaned_prefixes=$(echo $cleaned_prefixes | sed 's/contentstore.deleted,//')\
-          # remove `contentstore.deleted` from cleaned_prefixes if at the end of the list
-          cleaned_prefixes=$(echo $cleaned_prefixes | sed 's/contentstore.deleted//')
+          # None occurs when a file is at the root of the bucket or level 1 directory
+          # remove None from cleaned_prefixes with comma if in list
+          cleaned_prefixes=$(echo $cleaned_prefixes | sed 's/None,//')
+          # remove None from cleaned_prefixes if at the end of the list
+          cleaned_prefixes=$(echo $cleaned_prefixes | sed 's/None//')
+
 
           DIRS=""
           IFS=','
@@ -93,7 +95,4 @@ jobs:
             --set srcBucket=$SRC_BUCKET \
             --set "directories={${DIRS}}"
 
-          kubectl wait jobs -l name-prefix=migrate-s3 --for=condition=complete --timeout 10h
-
-      - name: Uninstall S3 Refresh chart
-        run: helm uninstall migrate-s3 --ignore-not-found
+          kubectl wait jobs -l name-prefix=migrate-s3 --for=condition=ready --timeout 10m

.gitignore

@@ -4,8 +4,11 @@
 templated/
 kustomize/base/charts/
 kustomize/**/charts/
-kustomize/**/output.yaml
+kustomize/**/helm_rendered_spec.yaml
+kustomize/**/kustomized_helm_rendered_spec.yaml
 kustomize/base/resources.yaml
 jobs/reindex/values-reindex-*.yaml
 completed.txt
 ids.json
+alfresco-content-services/
+s3-config/

README.md

@@ -1,102 +1,49 @@
-### Purpose
+# hmpps-delius-alfresco
 
-These example instsructions explain how you would customise one particular dependant Helm chart and publish a newer version to GitHub pages. Instructions assume GitHub pages are already configured for your repo; see the reference section below
+## Helm + Kustomize
 
-### Start services
-In order to start the alfresco-repository service, we need to make a valid  license available in the namespace. A secret containing the license needs to be created:
-```bash
-ACS_NAMESPACE=hmpps-delius-alfrsco-poc
-kubectl create secret generic alfresco-license \
-  --namespace $ACS_NAMESPACE \
-  --from-file /example/path/to/license/file.lic
-```
+This repository contains the Helm and Kustomize configuration for the Delius Alfresco deployment.
 
-Next We will need to ensure all services are up and running.
-Start k8s services by executing helm command _(Helm will complain if a random secret is not created)_
+Rather than using/modifying the Helm chart directly, we use Kustomize to overlay the Helm chart with our custom configuration.
+This allows us to keep the Helm chart as a dependency and only modify the configuration that we need to.
 
-```bash
-cd hmpps-delius-alfresco-poc/alfresco-content-services
-export SECRET=$(openssl rand -base64 20)
-export BUCKET_NAME=$(awk '{print substr($0, 0)}' <<< $(kubectl get secrets s3-bucket-output -o jsonpath='{.data.BUCKET_NAME}' | base64 -d))
-helm install alfresco-content-services . --values=./values.yaml \
---set s3connector.config.bucketName=$BUCKET_NAME \
---set global.tracking.sharedsecret=$SECRET
-```
+### Usage
+
+Note: we use taskfile to simplify the commands. You can install taskfile by running `brew install go-task/tap/go-task`.
+
+
+To deploy the Delius Alfresco stack, you can use the following command:
 
-### Check the chart file for dependent charts and pull the required version
-For the purpose of this demo, we will select the following service from the `Chart.yaml` file
-```yaml
-- condition: alfresco-sync-service.enabled
-  name: alfresco-sync-service
-  repository: https://alfresco.github.io/alfresco-helm-charts/
-  version: 4.1.0
 ```
+task helm_upgrade ENV=<dev|test|stage|preprod|prod> DEBUG=<true|false>
 ```
-1. Delete existing tar file for the sync service. There will be an error pulling the chart otherwise
-rm -rf charts/alfresco-sync-service-4.1.0.tgz
 
-2. Pull a particular version of `alfresco-sync-service` chart
-helm pull alfresco-sync-service --repo  https://alfresco.github.io/alfresco-helm-charts --version 4.1.0 -d charts --untar
+This will deploy the Delius Alfresco stack to the specified environment.
+The `DEBUG` flag can be used to enable debug mode, which will enable helm verbose logging + output the templated,
+rendered and kustomized manifests to the environment directory.
 
-3. The above command will pull a tar file called `charts/alfresco-sync-service-4.1.0.tgz` and then untar it into a directory called `alfresco-sync-service`. Delete the tar file
-rm -rf charts/alfresco-sync-service-4.1.0.tgz
-```
 
-### Modify charts
-
-1. Change the chart version in the newly pulled chart. For example change is from `4.1.0` to `4.1.1`
-2. Make your changes and then test them by upgrading Helm release
-   ```
-   - export SECRET=$(awk '{print substr($0, 19)}' <<< $(kubectl get secrets alfresco-content-services-alfresco-repository-properties-secret -o jsonpath='{.data.alfresco-global\.properties}' | base64 -d))
-   - export BUCKET_NAME=$(awk '{print substr($0, 0)}' <<< $(kubectl get secrets s3-bucket-output -o jsonpath='{.data.BUCKET_NAME}' | base64 -d))
-   - helm upgrade alfresco-content-services . --values=./values.yaml --set s3connector.config.bucketName=$BUCKET_NAME --set global.tracking.sharedsecret=$SECRET
-   - NOTE: For the release upgrade, use the existing secret. You will otherwise have to restart pods consuming those secrets
-   ```
-4. Once satisfied with your changes, create a package and add it to the docs directory
-   - "helm package charts/alfresco-sync-service -d ../docs"
-5. Create / update an index file in docs directory
-   - "helm repo index ../docs --url https://ministryofjustice.github.io/hmpps-delius-alfresco-poc"
-
-
-### Update the lock file and commit changes
-Locate the `Chart.yaml` file and modify the repository URL and version. It should now look like the code snippet below after the change:
-```yaml
-- condition: alfresco-sync-service.enabled
-  name: alfresco-sync-service
-  repository: https://ministryofjustice.github.io/hmpps-delius-alfresco-poc/
-  version: 4.1.1
-```
+### Configuration
 
-1. Delete `charts/alfresco-sync-service` directory as it is no longer needed
-2. Push your changes / docs directory to the feature branch
-3. Update your GitHub pages settings so that the `source branch` is pointing to your feature branch
-4. Update helm dependencies which will pull the updated charts and will update the lock file
-   - `helm dependency update .`
-5. Push the lock file and charts dirctory to the feature branch and get merge approval
-6. Merge into main branch
-7. Update your GitHub pages settings so that the `source branch` is pointing to your main branch
-8. Upgrade the helm release for the changes to be updated in kubernetes cluster
-   ```
-   - export SECRET=$(awk '{print substr($0, 19)}' <<< $(kubectl get secrets alfresco-content-services-alfresco-repository-properties-secret -o jsonpath='{.data.alfresco-global\.properties}' | base64 -d))
-   - helm upgrade alfresco-content-services . --values=./values.yaml --set global.tracking.sharedsecret=$SECRET
-   - NOTE: For the release upgrade, use the existing secret. You will otherwise have to restart pods consuming those secrets
-   ```
-
-### Alternatively, pull a particular chart either directly from repository URL or by adding it in the local repo
-
-1. Pull a chart with a particular version direcly from the GitHub pages
-- `helm pull alfresco-sync-service --repo  https://ministryofjustice.github.io/hmpps-delius-alfresco-poc/ --version 4.1.1 -d charts --untar`
-
-2. Or add the updated chart in a local helm repo
-```
-helm repo add alfresco-sync-service https://ministryofjustice.github.io/hmpps-delius-alfresco-poc/
+1. Helm values
+The base helm values are stored in the `kustomize/base/values.yaml` file.
+Each environment has its own values file, which is stored in the `kustomize/environments/<env>/values.yaml` file.
+These values are combined when deploying the stack, with the environment values taking precedence.
 
-helm search repo alfresco-sync-service
-NAME                                            CHART VERSION   APP VERSION     DESCRIPTION
-alfresco-sync-service/alfresco-sync-service     4.1.1           3.9.0           Alfresco Sync Service
-```
+2. Kustomize
+The kustomize overlays are stored in the `kustomize/environments/<env>` directory.
+These overlays are applied to the Helm chart's resources to modify the configuration as needed for the environment.
+
+
+### Secrets
+
+A number of secrets are required to deploy the Delius Alfresco stack. Some of these are set by the cloud-platform-environments repository, while others are set manually.
 
-### References to various docs that explain how to set up GitHub pages and how to publish Helm charts
-- [The Chart Repository Guide](https://helm.sh/docs/topics/chart_repository/#github-pages-example)
-- [Chart Releaser Action to Automate GitHub Page Charts ](https://helm.sh/docs/howto/chart_releaser_action/#github-actions-workflow)
-- [Example on how to publish a chart on GitHub pages](https://github.com/technosophos/tscharts)
+Table:
+| Secret Name | Description | Set By | example/required keys |
+| --- | --- | --- | --- |
+| amazon-mq-broker-secret | The secret for the Amazon MQ broker | cloud-platform-environments |  see [cloud-platform-environments](https://github.com/ministryofjustice/cloud-platform-environments/blob/7968f9c66f6914d33db35b68209c55b2dcb25d7d/namespaces/live.cloud-platform.service.justice.gov.uk/hmpps-delius-alfresco-stage/resources/amq.tf#L218) |
+| alfresco-license | The Alfresco license file | manual | `<alfresco-license-file-name> : <base64-encoded-alfresco-license-file>`
+| legacy-rds-instance | The RDS instance for the legacy Delius Alfresco stack | manual | `DATABASE_NAME: <database-name>, DATABASE_USERNAME: <database-username>, DATABASE_PASSWORD: <database-password>, RDS_INSTANCE_ADDRESS: <rds-instance-address>` |
+| rds-instance-outpur | The RDS instance for the CP Delius Alfresco stack | cloud-platform-environments | see [cloud-platform-environments](https://github.com/ministryofjustice/cloud-platform-environments/blob/7968f9c66f6914d33db35b68209c55b2dcb25d7d/namespaces/live.cloud-platform.service.justice.gov.uk/hmpps-delius-alfresco-stage/resources/rds.tf#L35) |
+| quay-registry-secret | The secret for the Quay registry | manual | `.dockerconfigjson: {"auths":{"quay.io":{"username":"<quay-username>","password":"<quay-password>","email":"<quay-email>","auth":"<base64-encoded-auth>"}}}` |