Kargo Integration

[jeffmccune]

Next Steps

Manage the resources in the Quickstart

• Credentials - Need to manually create a github app secret as documented here and add to mkcert -CAROOT/kargo.yaml. Also need to manage the namespace. Not ideal.
• Projects - Managed like namespaces, if the kargo.akuity.io/project: "true" label is set on the namespace.
• Warehouse
• Stages dev, test, stage, prod
• Re-orient to 3 projects: bank-security, bank-backend, bank-web.
• Env stages are an aspect of each project.
• Kargo Project Namespace for each Project.
• ArgoCD Update.

Next step, resolve:

failed to run step "argocd-update": error getting Argo CD Application "dev-bank-web-frontend" in namespace "argocd": Argo CD Application "dev-bank-web-frontend" in namespace "argocd" does not permit mutation by Kargo Stage dev in namespace dev-bank-frontend

Gotchas

• GitHub App private key needs to be double base64 encoded. See GitHub Apps Credentials - error decoding private key, cannot figure out correct format akuity/kargo#3055

Summary

With Kargo v1.1.0 integrate by having holos write a kustomization.yaml file artifact. Integrate with Kargo using:

1. git-clone
2. kustomize-set-image
3. kustomize-build
4. git-commit
5. git-push

Need to prototype this out for one of our own internal apps. Do it with holos server.

The downside is holos no longer produces the final manifests, kargo does, but the fully rendered manifests are still in Git and this integration can occur on a component by component basis.

Custom Promotion Steps

The ideal integration is with the External/Custom Promotion Steps planned for Kargo v1.3.0.

Refer to akuity/kargo#2996

Issues / Discussions

Track work toward custom/external pipeline steps.

• ui: support for new steps yaml-update and http
• Kargo integration with Holos
• Multiple pipelines
• File/Directory/Patch based promotion
• refactor: promotion mechanisms --> promotion directives

Might be possible to fiddle JSON using CEL, I asked Krancour about this here:
akuity/kargo#1250 (comment)

[jeffmccune]

Got this mostly working up to the argocd-update step. At this point it became clear we need to create an additional namespace for the project.

A Kargo Project has a special Namespace of the same name to collect all of the kargo resources. See Working with Projects

Each Kargo project is represented by a cluster-scoped Kubernetes resource of type Project. Reconciliation of such a resource effects all boilerplate project initialization, including the creation of a specially-labeled Namespace with the same name as the Project. All resources belonging to a given Project should be grouped together in that Namespace.

This is close to the current Holos definition of a Project, but not quite the same. At present, Bank of Holos is split up into 3 projects per each of the 4 environments, for 12 projects total.

If we were to re-orient our model to align with Kargo, then we would create 3 projects with environment stages nested within them:

1. bank-security
2. bank-web
3. bank-backend

This could still fit our alignment with "projects are security boundaries" with the additional consideration that environments may further restrict access, e.g. prod bank web is still part of the bank-web project, but has restricted access compared to dev-bank-web.

[jeffmccune]

• Remove the bank-security Kargo project
• Move Warehouse and Stages into the respective frontend and backend project namespaces.
• Progress dev to test to stage in nonprod
• Progress east to west in prod

Or maybe don't? It's nice to see everything in one place.

[jeffmccune]

Implemented the prod/nonprod split and promotion stages from dev, test, staging and prod east to west in holos-run/bank-of-holos#11

Recorded a quick video about this at https://www.youtube.com/watch?v=m0bpQugSbzA

Tagged in v0.6.2 and documented at https://github.com/holos-run/bank-of-holos/blob/v0.6.2/docs/kargo.md