Skip to content

Latest commit

 

History

History
142 lines (121 loc) · 5.46 KB

20220615-sidecarset-patch-pod-metadata.md

File metadata and controls

142 lines (121 loc) · 5.46 KB
title authors reviewers creation-date last-updated status
SidecarSetPatchPodMetadata
@zmberg
@furykerry
@FillZpp
2021-06-15
2021-08-10
implementable

SidecarSet Patch Pod Metadata

Table of Contents

A table of contents is helpful for quickly jumping to sections of a proposal and for highlighting any additional information provided beyond the standard proposal template. Tools for generating a table of contents from markdown are available.

Motivation

Some sidecar containers may require special configurations that take effect via annotations/labels. Thus, sidecarSet should support inject or in-place update these configurations.

User Story:

  • log-agent sidecar container inject pod annotation[oom-score]='{"log-agent": 1}' to set sidecar oom score.

Proposal

SidecarSet support patch pod annotations/labels, as follows:

Api Definition

// SidecarSetSpec defines the desired state of SidecarSet
type SidecarSetSpec struct {
    // SidecarSet support to inject & in-place update metadata in pod.
    PatchPodMetadata []SidecarSetPatchPodMetadata `json:"patchPodMetadata,omitempty"`
}

type SidecarSetPatchPodMetadata struct {
    // annotations
    Annotations map[string]string `json:"annotations,omitempty"`

    // labels map[string]string `json:"labels,omitempty"`
    // patch pod metadata policy, Default is "Retain"
    PatchPolicy SidecarSetPatchPolicyType `json:"patchPolicy,omitempty"`
}

type SidecarSetPatchPolicyType string

var (
    // SidecarSetRetainPatchPolicy indicates if PatchPodFields conflicts with Pod,
    // will ignore PatchPodFields, and retain the corresponding fields of pods.
    // SidecarSet webhook cannot allow the conflict of PatchPodFields between SidecarSets under this policy type.
    // Note: Retain is only supported for injection, and the Metadata will not be updated when upgrading the Sidecar Container in-place.
    SidecarSetRetainPatchPolicy SidecarSetPatchPolicyType = "Retain"

    // SidecarSetOverwritePatchPolicy indicates if PatchPodFields conflicts with Pod,
    // SidecarSet will apply PatchPodFields to overwrite the corresponding fields of pods.
    // SidecarSet webhook cannot allow the conflict of PatchPodFields between SidecarSets under this policy type.
    // Overwrite support to inject and in-place metadata.
    SidecarSetOverwritePatchPolicy SidecarSetPatchPolicyType = "Overwrite"

    // SidecarSetMergePatchJsonPatchPolicy indicate that sidecarSet use application/merge-patch+json to patch annotation value,
    // for example, A patch annotation[oom-score] = '{"log-agent": 1}' and B patch annotation[oom-score] = '{"envoy": 2}'
    // result pod annotation[oom-score] = '{"log-agent": 1, "envoy": 2}'
    // MergePatchJson support to inject and in-place metadata.
    SidecarSetMergePatchJsonPatchPolicy SidecarSetPatchPolicyType = "MergePatchJson"
)

Permission Control

SidecarSet should not modify any configuration outside the sidecar container from permission perspective. Metadata, as an important configuration of Pod, should not be modified by sidecarSet by default.

Objectively, sidecar does need to have some annotations or labels injected into the pod as well. In order to meet the needs of sidecar and security considerations. if sidecarSet needs to modify the metadata, it needs to be whitelisted in kruise configmap which is maintained by the system administrator.

apiVersion: v1
kind: ConfigMap
metadata:
    name: kruise-configuration
    namespace: kruise-system
data:
  "SidecarSet_PatchPodMetadata_WhiteList": |
    type SidecarSetPatchMetadataWhiteList struct {
        Rules []SidecarSetPatchMetadataWhiteRule `json:"rules"`
    }

    type SidecarSetPatchMetadataWhiteRule struct {
        // selector sidecarSet against labels
        // If selector is nil, assume that the rules should apply for every sidecarSets
        Selector *metav1.LabelSelector `json:"selector,omitempty"`
        // must be regular expressions
        AllowedAnnotationKeyExprs []string `json:"allowedAnnotationKeyExprs"`
    }

for example:

apiVersion: v1
kind: ConfigMap
metadata:
    name: kruise-configuration
    namespace: kube-system
data:
    "SidecarSet_PatchPodMetadata_WhiteList": |
        {
            "rules": [
                {
                    // No selector is configured to take effect for all SidecarSets
                    "allowedAnnotationKeyExprs": [
                        "^oom-score$",
                    ]
                },
                {
                    // selector sidecarSet based on labels
                    "selector":  {
                        "matchLabels": {
                            "sidecar": "sidecar-demo"
                        }
                    }
                    "allowedAnnotationKeyExprs": [
                        "key.*",
                    ]
                },
            ]
        }

feature-gates

"SidecarSet_PatchPodMetadata_WhiteList" is mainly for security reasons. If the user's business cluster scenario is relatively simple, the whitelist method may not be used.

$ helm install kruise https://... --set featureGates="SidecarSetPatchPodMetadataDefaultsAllowed=true"

Note: The above feature-gate will not verify any sidecarSet patch key, so users need to ensure security.