feat(ci): configure in cluster dagger engine

[Smana]

PR Type

Enhancement, Configuration changes, Documentation

---

Description

• Added new data sources for Kubernetes manifests in EKS module.
• Refactored Karpenter manifests to use template files.
• Added variable for defining Karpenter node pool limits.
• Updated issue and enhancement templates.
• Replaced Task installation with Dagger module for Terraform/Opentofu validation in CI.
• Updated CI documentation and added Dagger example.
• Commented out Loki and Vector-Agent in observability stack.
• Updated roleId for Vault AppRole in cert-manager.
• Added example configuration for Karpenter limits.
• Added manifests for Karpenter default and IO node pools.
• Added Dagger engine configuration, deployment, and related resources.
• Added HelmRelease for GHA runner scale set controller and Dagger GHA runner scale set.
• Updated HelmRelease, ExternalSecret, and NetworkPolicy for GHA runner scale set.

---

Changes walkthrough 📝

	Relevant files
Enhancement	17 files data.tf Add data sources for Kubernetes manifests in EKS module    terraform/eks/data.tf Added new data sources for Kubernetes manifests. +9/-0      karpenter.tf Refactor Karpenter manifests to use template files              terraform/eks/karpenter.tf Refactored kubectl_manifest resource to use template files. Removed inline YAML definitions. +20/-64  variables.tf Add variable for Karpenter node pool limits                            terraform/eks/variables.tf Added new variable karpenter_limits for defining limits per node pool. +11/-0    default-ec2nc.yaml Add EC2NodeClass manifest for default node pool                    terraform/eks/kubernetes-manifests/karpenter/default-ec2nc.yaml Added EC2NodeClass manifest for default node pool. +15/-0    default-nodepool.yaml Add NodePool manifest for default node pool                            terraform/eks/kubernetes-manifests/karpenter/default-nodepool.yaml Added NodePool manifest for default node pool. +29/-0    io-ec2nc.yaml Add EC2NodeClass manifest for IO node pool                              terraform/eks/kubernetes-manifests/karpenter/io-ec2nc.yaml Added EC2NodeClass manifest for IO node pool. +19/-0    io-nodepool.yaml Add NodePool manifest for IO node pool                                      terraform/eks/kubernetes-manifests/karpenter/io-nodepool.yaml Added NodePool manifest for IO node pool. +39/-0    configmap.yaml Add ConfigMap for Dagger engine configuration                        tooling/base/dagger-engine/configmap.yaml Added ConfigMap for Dagger engine configuration. +10/-0    deployment.yaml Add Deployment for Dagger engine                                                  tooling/base/dagger-engine/deployment.yaml Added Deployment for Dagger engine. +104/-0  kustomization.yaml Add kustomization file for Dagger engine                                  tooling/base/dagger-engine/kustomization.yaml Added kustomization file for Dagger engine. +10/-0    network-policy.yaml Add NetworkPolicy for Dagger engine                                            tooling/base/dagger-engine/network-policy.yaml Added NetworkPolicy for Dagger engine. +39/-0    pdb.yaml Add PodDisruptionBudget for Dagger engine                                tooling/base/dagger-engine/pdb.yaml Added PodDisruptionBudget for Dagger engine. +9/-0      service.yaml Add Service for Dagger engine                                                        tooling/base/dagger-engine/service.yaml Added Service for Dagger engine. +13/-0    controller-helmrelease.yaml Add HelmRelease for GHA runner scale set controller            tooling/base/gha-runners/controller-helmrelease.yaml Added HelmRelease for GHA runner scale set controller. +30/-1    dagger-scale-set-helmrelease.yaml Add HelmRelease for Dagger GHA runner scale set                    tooling/base/gha-runners/dagger-scale-set-helmrelease.yaml Added HelmRelease for Dagger GHA runner scale set. +25/-0    kustomization.yaml Add Dagger scale set HelmRelease to kustomization                tooling/base/gha-runners/kustomization.yaml Added Dagger scale set HelmRelease to kustomization. +1/-0      kustomization.yaml Add Dagger engine and GHA runners to kustomization              tooling/mycluster-0/kustomization.yaml Added Dagger engine and GHA runners to kustomization. +6/-4
Documentation	4 files bug_report.md Update issue template title format                                              .github/ISSUE_TEMPLATE/bug_report.md Updated issue template title format. +1/-1      enhancement.md Update enhancement template title format and structure      .github/ISSUE_TEMPLATE/enhancement.md Updated enhancement template title format. Moved benefits section under motivation. +2/-4      README.md Update CI documentation and add Dagger example                      README.md Updated CI section to reflect transition to Dagger. Added example for Dagger with self-hosted runners. +38/-17  README.md Add example configuration for Karpenter limits                      terraform/eks/README.md Added example configuration for karpenter_limits. +12/-0
Configuration changes	8 files ci.yaml Use Dagger module for Terraform/Opentofu validation in CI .github/workflows/ci.yaml Replaced Task installation with Dagger module for Terraform/Opentofu validation. +8/-8      observability.yaml Comment out Loki and Vector-Agent in observability stack  clusters/mycluster-0/observability.yaml Commented out Loki and Vector-Agent HelmRelease resources. +8/-8      kustomization.yaml Comment out Loki and Vector-Agent in observability kustomization observability/mycluster-0/kustomization.yaml Commented out Loki and Vector-Agent resources. +3/-2      vault-clusterissuer.yaml Update roleId for Vault AppRole in cert-manager                    security/base/cert-manager/vault-clusterissuer.yaml Updated roleId for Vault AppRole authentication. +1/-1      default-scale-set-helmrelease.yaml Update HelmRelease for default GHA runner scale set            tooling/base/gha-runners/default-scale-set-helmrelease.yaml Updated HelmRelease for default GHA runner scale set. +1/-2      externalsecret.yaml Update ExternalSecret for GHA runner scale set                      tooling/base/gha-runners/externalsecret.yaml Updated ExternalSecret for GHA runner scale set. +2/-2      network-policy.yaml Update NetworkPolicy for Dagger GHA runner scale set          tooling/base/gha-runners/network-policy.yaml Updated NetworkPolicy to include Dagger GHA runner scale set. +14/-3    source.yaml Update HelmRepository for GHA runner scale set                      tooling/base/gha-runners/source.yaml Updated HelmRepository for GHA runner scale set. +0/-1

---

💡 PR-Agent usage:
Comment /help on the PR to get a list of all available PR-Agent tools and their descriptions

[github-actions]

PR Reviewer Guide 🔍

⏱️ Estimated effort to review [1-5]	3
🧪 Relevant tests	No
🔒 Security concerns	No
⚡ Key issues to review	Configuration Consistency: Ensure that the templating and variable substitutions in the Kubernetes manifests are correctly implemented and consistent across different environments.
	Resource Limits: Review the specified resource limits and requests in the HelmRelease for the Dagger Engine to ensure they are appropriate for the expected workload.
	Dependency Management: Verify that the dependencies, such as the HelmRepository and Kustomization resources, are correctly configured and that the versions specified are compatible with each other.

[github-actions]

PR Code Suggestions ✨

Category	Suggestion	Score
Best practice	Add additional dependencies to the kubectl_manifest resource to ensure all necessary resources are ready before application Consider adding a depends_on attribute for the kubectl_manifest "karpenter" to ensure that the resources it depends on are created or updated before this manifest is applied. This can help avoid potential race conditions where the manifest is applied before the necessary resources are ready. terraform/eks/karpenter.tf [22-44] resource "kubectl_manifest" "karpenter" { for_each = { for file_name in flatten([ data.kubectl_filename_list.karpenter_default.matches, data.kubectl_filename_list.karpenter_io.matches ]) : file_name => file_name } yaml_body = templatefile( each.key, { cluster_name = module.eks.cluster_name, env = var.env, karpenter_node_iam_role_name = module.karpenter.node_iam_role_name default_nodepool_cpu_limits = var.karpenter_limits.default.cpu default_nodepool_memory_limits = var.karpenter_limits.default.memory io_nodepool_cpu_limits = var.karpenter_limits.io.cpu io_nodepool_memory_limits = var.karpenter_limits.io.memory } ) depends_on = [ - helm_release.karpenter + helm_release.karpenter, + data.kubectl_filename_list.karpenter_default, + data.kubectl_filename_list.karpenter_io ] } Suggestion importance[1-10]: 9 Why: The suggestion correctly identifies a potential race condition and provides a solution by adding dependencies to ensure the necessary resources are ready before applying the manifest. This is a best practice for improving the reliability of the deployment process.	9

[Smana]

/describe

[github-actions]

PR Description updated to latest commit (17932c2)

[Smana]

/describe

[github-actions]

PR Description updated to latest commit (f282b78)

[Smana]

/describe

[github-actions]

PR Description updated to latest commit (b4cda64)