-
Notifications
You must be signed in to change notification settings - Fork 179
Commit
This commit does not belong to any branch on this repository, and may belong to a fork outside of the repository.
Signed-off-by: Niklas Wagner <[email protected]>
- Loading branch information
1 parent
5efd25a
commit c82a0ed
Showing
1 changed file
with
93 additions
and
0 deletions.
There are no files selected for viewing
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,93 @@ | ||
| --- | ||
| hip: 9999 | ||
| title: "Custom resource ordering" | ||
| authors: [ "Niklas Wagner" ] | ||
| created: "2022-01-10" | ||
| type: "feature" | ||
| status: "draft" | ||
| --- | ||
|
|
||
| ## Abstract | ||
|
|
||
| Helm currently has a hard-coded order of all Kubernetes resources when installing or uninstalled. Since Kubernetes allows creating own resource types via CustomResourceDefinitions, there need to be a way for a user to specify the installation order of custom resources. | ||
|
|
||
| This proposal describes how to allow users to control the order of installation of custom resources. | ||
|
|
||
| ## Motivation | ||
|
|
||
| When installing a Helm chart, Helm will automatically sort the resources in a specific way to avoid conflicts from the Kubernetes API. This is done by ordering the resources based on a predefined order, hard-coded in the Helm code [here](https://github.com/helm/helm/blob/39ca699ca790e02ba36753dec6ba4177cc68d417/pkg/releaseutil/kind_sorter.go#L31-L66). All resource types that are not in the predefined order will be placed at the end of the installation order. | ||
|
|
||
| Custom resources often need to be installed in a specific order. For example, the AWS custom resource `SecurityGroupPolicy` needs to be installed before any Pod starts. This is because the `SecurityGroupPolicy` resource configures the network policy for the pods. | ||
|
|
||
| This proposal allows users to specify the order of installation of custom resources by adding a new annotation to the custom resource. The annotation is similar to the already existing [chart hooks annotations](https://helm.sh/docs/topics/charts_hooks/). | ||
|
|
||
| ## Rationale | ||
|
|
||
| Helm already uses many annotations to control how resources are deployed. For example, the [chart hooks annotations](https://helm.sh/docs/topics/charts_hooks/) to run specific resources before or after installing a chart. Therefore, it makes sense to add a new annotation to control the order of installation of custom resources. | ||
|
|
||
| ## Specification | ||
|
|
||
| ### Overview | ||
|
|
||
| The new annotation is called `helm.sh/install-before`. The value contains a comma separated list of resource types. Helm then makes sure that the custom resource is installed before the resources specified in the list. | ||
|
|
||
| A custom resource would look like this: | ||
|
|
||
| ```yaml | ||
| apiVersion: vpcresources.k8s.aws/v1beta1 | ||
| kind: SecurityGroupPolicy | ||
| metadata: | ||
| name: niklas-debug | ||
| labels: | ||
| helm.sh/chart: generic-service-0.5.0 | ||
| annotations: | ||
| helm.sh/install-before: "Deployment,Statefulset,DaemonSet" | ||
| spec: | ||
| - <snip> | ||
| ``` | ||
| In this case, the `SecurityGroupPolicy` resource will be installed before any `Deployment`, `Statefulset` and `DaemonSet` resources are installed. | ||
|
|
||
| ### How it works? | ||
|
|
||
| The value of the annotation is parsed and split into a list of resource types. The parsed resources are then matched against the already existing installation order. The sort algorithm then makes sure that the custom resource is placed before all matching resources in the installation order. | ||
|
|
||
| The resource type is case-sensitive. All specified resource types must exist in the hard-coded installation order. Custom resources are currently not supported. If the resource type couldn't be found in the resource list, it will be gracefully ignored. | ||
|
|
||
| Multiple resource types can be specified in the annotation. This makes sure that users don't rely on the current hard-coded order of resources and allow Helm maintainer to change the order in the future without breaking existing installations. | ||
|
|
||
| ## Backwards compatibility | ||
|
|
||
| There should be no impact to backwards compatibility. | ||
|
|
||
| * Older helm version will ignore the annotations and install the custom resources at the end of the installation order. | ||
| * Newer helm version will install custom resources without the annotation at the end of the installation order. | ||
|
|
||
| ## Security implications | ||
|
|
||
| None. | ||
|
|
||
| ## How to teach this | ||
|
|
||
| The feature would be added to the documentation and mentioned in the release notes. | ||
|
|
||
| ## Reference implementation | ||
|
|
||
| A working proof-of-concept implementation can be found here: | ||
|
|
||
| * https://github.com/helm/helm/pull/9534 | ||
|
|
||
| ## Rejected ideas | ||
|
|
||
| This is the initial implementation of the feature, and as such there have been no other ideas to be rejected. | ||
|
|
||
| ## Open issues | ||
|
|
||
| There are already numerous issues for this exact problem, dating back to early 2019: | ||
|
|
||
| * https://github.com/helm/helm/issues/8439 | ||
| * https://github.com/helm/helm/issues/8291 | ||
| * https://github.com/helm/helm/issues/7072 | ||
| * https://github.com/helm/helm/issues/6851 | ||
| * https://github.com/helm/helm/issues/5916 | ||
| * https://github.com/helm/helm/issues/5642 |