Merge branch 'main' into policyflag

Signed-off-by: Sanskar Gurdasani <[email protected]>
kyverno · Apr 22, 2024 · 1ea2d9c · 1ea2d9c
2 parents a268613 + da53d26
commit 1ea2d9c
Show file tree

Hide file tree

Showing 24 changed files with 1,275 additions and 3 deletions.
diff --git a/go.mod b/go.mod
@@ -9,7 +9,7 @@ require (
 	github.com/spf13/cobra v1.8.0
 	go.uber.org/multierr v1.11.0
 	google.golang.org/genproto/googleapis/rpc v0.0.0-20240227224415-6ceb2ff114de
-	google.golang.org/grpc v1.63.0
+	google.golang.org/grpc v1.63.2
 	k8s.io/apimachinery v0.29.3
 )
 

diff --git a/go.sum b/go.sum
@@ -324,10 +324,10 @@ google.golang.org/genproto/googleapis/api v0.0.0-20240227224415-6ceb2ff114de/go.
 google.golang.org/genproto/googleapis/rpc v0.0.0-20240227224415-6ceb2ff114de h1:cZGRis4/ot9uVm639a+rHCUaG0JJHEsdyzSQTMX+suY=
 google.golang.org/genproto/googleapis/rpc v0.0.0-20240227224415-6ceb2ff114de/go.mod h1:H4O17MA/PE9BsGx3w+a+W2VOLLD1Qf7oJneAoU6WktY=
 google.golang.org/grpc v1.18.0/go.mod h1:6QZJwpn2B+Zp71q/5VxRsJ6NXXVCE5NRUHRo+f3cWCs=
-google.golang.org/grpc v1.63.0 h1:WjKe+dnvABXyPJMD7KDNLxtoGk5tgk+YFWN6cBWjZE8=
-google.golang.org/grpc v1.63.0/go.mod h1:WAX/8DgncnokcFUldAxq7GeB5DXHDbMF+lLvDomNkRA=
 google.golang.org/protobuf v1.26.0-rc.1/go.mod h1:jlhhOSvTdKEhbULTjvd4ARK9grFBp09yW+WbY/TyQbw=
 google.golang.org/protobuf v1.26.0/go.mod h1:9q0QmTI4eRPtz6boOQmLYwt+qCgq0jsYwAQnmE0givc=
+google.golang.org/grpc v1.63.2 h1:MUeiw1B2maTVZthpU5xvASfTh3LDbxHd6IJ6QQVU+xM=
+google.golang.org/grpc v1.63.2/go.mod h1:WAX/8DgncnokcFUldAxq7GeB5DXHDbMF+lLvDomNkRA=
 google.golang.org/protobuf v1.33.0 h1:uNO2rsAINq/JlFpSdYEKIZ0uKD/R9cpdv0T+yoGwGmI=
 google.golang.org/protobuf v1.33.0/go.mod h1:c6P6GXX6sHbq/GpV6MGZEdwhWPcYBgnhAHhKbcUYpos=
 gopkg.in/check.v1 v0.0.0-20161208181325-20d25e280405/go.mod h1:Co6ibVJAznAaIkqp8huTwlJQCZ016jof/cbN4VW5Yz0=

diff --git a/sidecar-injector/Dockerfile b/sidecar-injector/Dockerfile
@@ -0,0 +1,11 @@
+FROM golang:1.22 as build
+RUN go install golang.org/x/lint/golint@latest
+WORKDIR /build
+COPY . ./
+RUN CGO_ENABLED=0 GOOS=linux go build -o sidecar-injector
+
+FROM scratch
+WORKDIR /
+COPY --from=build /build/sidecar-injector /
+
+ENTRYPOINT ["/sidecar-injector"]
diff --git a/sidecar-injector/README.md b/sidecar-injector/README.md
@@ -0,0 +1,202 @@
+# Kyverno Envoy Sidecar Injector 
+
+Uses [MutatingAdmissionWebhook Controller](https://kubernetes.io/docs/reference/access-authn-authz/admission-controllers/#mutatingadmissionwebhook) in Kubernetes to inject kyverno-envoy-plugin sidecar into newly created pods. This injection occurs at pod creation time, targeting pods that have the label `kyverno-envoy-sidecar/injection=enabled`. By introducing this sidecar, we can enforce policies on all incoming HTTP requests and make external authorization decisions to the targeted pod without modifying the primary application code/containers.
+
+
+## Prerequisites
+
+Kubernetes 1.16.0 or above with the `admissionregistration.k8s.io/v1` API enabled.
+Verify that by the following command:
+```bash
+ ~$ kubectl api-versions | grep admissionregistration.k8s.io/v1
+```
+The result should be:
+```bash
+admissionregistration.k8s.io/v1
+```
+## Installation  
+
+#### Dedicated Namespace
+
+Create a namespace `kyverno-envoy-sidecar-injector`, where you will deploy the Kyverno Envoy Sidecar Injector Webhook components.
+
+```bash
+ ~$ kubectl create namespace kyverno-envoy-sidecar-injector
+```
+
+#### Deploy Sidecar Injector
+
+1. Create a signed cert/key pair and store it in a Kubernetes `secret` that will be consumed by sidecar injector deployment 
+
+    Generate cert/key pair with openssl 
+    ```bash
+     ~$ openssl req -new -x509  \
+        -subj "/CN=kyverno-envoy-sidecar.kyverno-envoy-sidecar-injector.svc" \
+        -addext "subjectAltName = DNS:kyverno-envoy-sidecar.kyverno-envoy-sidecar-injector.svc" \
+        -nodes -newkey rsa:4096 -keyout tls.key -out tls.crt 
+    ```
+    Now apply below command to create `secret`  
+    ```bash
+     ~$ kubectl create secret generic kyverno-envoy-sidecar-certs \
+        --from-file tls.crt=tls.crt \
+        --from-file tls.key=tls.key \
+        --dry-run=client -n kyverno-envoy-sidecar-injector -oyaml > secret.yaml 
+    ```  
+    Apply the secret
+    ```bash
+    ~$ kubectl apply -f secret.yaml
+    ```
+
+2. Run the script to Patch the `Mutating Webhook Configuration` with the CA bundle extracted from the `secret` created in the previous step and apply the MutatingWebhookConfiguration changes: 
+
+```bash
+ ~$ ./manifests/create-mutating-webhook.sh
+```
+3. To Inject the Kyverno Envoy Sidecar, Create this configmap of name `kyverno-envoy-sidecar` in `kyverno-envoy--sidecar-injector` namespace. If their is requirement of multiple policy files, you can add more `--policy` flags and then add them in the `policy-files` configmap.
+
+```bash
+kubectl apply -f - <<EOF
+apiVersion: v1
+kind: ConfigMap
+metadata:
+  name: kyverno-envoy-sidecar
+  namespace: kyverno-envoy-sidecar-injector
+data:
+  sidecars.yaml: |
+    - name: kyverno-envoy-sidecar
+      containers:
+      - image: sanskardevops/plugin:0.0.25
+        imagePullPolicy: IfNotPresent
+        name: ext-authz
+        ports:
+        - containerPort: 8000
+        - containerPort: 9000
+        args:
+        - "serve"
+        - "--policy=/policies/policy.yaml"
+        volumeMounts:
+        - name: policy-files
+          mountPath: /policies
+      volumes:
+      - name: policy-files
+        configMap:
+          name: policy-files      
+EOF          
+```
+
+4. Deploy resources
+```bash
+ ~$ kubectl apply -f ./manifests/rbac.yaml
+ ~$ kubectl apply -f ./manifests/deployment.yaml
+ ~$ kubectl apply -f ./manifests/service.yaml
+```
+
+#### Verify Sidecar Injector Installation
+
+1. The sidecar injector should be deployed in the `kyverno-envoy-sidecar-injector` namespace:
+
+```bash
+ ~$ kubectl -n kyverno-envoy-sidecar-injector get all
+```
+```bash
+~$ kubectl -n  kyverno-envoy-sidecar-injector get all 
+NAME                                        READY   STATUS    RESTARTS   AGE
+pod/kyverno-envoy-sidecar-976c94445-2l66q   1/1     Running   0          46s
+
+NAME                            TYPE        CLUSTER-IP     EXTERNAL-IP   PORT(S)   AGE
+service/kyverno-envoy-sidecar   ClusterIP   10.96.137.93   <none>        443/TCP   3m11s
+
+NAME                                    READY   UP-TO-DATE   AVAILABLE   AGE
+deployment.apps/kyverno-envoy-sidecar   1/1     1            1           46s
+
+NAME                                              DESIRED   CURRENT   READY   AGE
+replicaset.apps/kyverno-envoy-sidecar-976c94445   1         1         1       46s
+
+```
+2. Now create a pod with the label `kyverno-envoy-sidecar/injection=enabled` in any namespace other than `kyverno-envoy-sidecar-injector`. But before we have to apply configmap `policy-files` in the same namespace where we will create the pod.
+```bash
+kubectl apply -f - <<EOF
+apiVersion: v1
+kind: ConfigMap
+metadata:
+  name: policy-files
+  namespace: default
+data:
+  policy.yaml: |
+    apiVersion: json.kyverno.io/v1alpha1
+    kind: ValidatingPolicy
+    metadata:
+      name: check-dockerfile
+    spec:
+      rules:
+        - name: deny-external-calls
+          assert:
+            all:
+            - message: "HTTP calls are not allowed"
+              check:
+                request:
+                    http:
+                        method: GET
+                        headers:
+                            authorization:
+                                (base64_decode(split(@, ' ')[1])): 
+                                    (split(@, ':')[0]): alice
+                        path: /foo    
+EOF          
+```
+
+Now create a pod with the label `kyverno-envoy-sidecar/injection=enabled` in the default namespace:
+
+```yaml
+apiVersion: apps/v1
+kind: Deployment
+metadata:
+  name: nginx
+  namespace: default
+  labels:
+    app.kubernetes.io/name: nginx
+spec:
+  replicas: 1
+  selector:
+    matchLabels:
+      app.kubernetes.io/name: nginx
+  template:
+    metadata:
+      labels:
+        kyverno-envoy-sidecar/injection: enabled
+        app.kubernetes.io/name: nginx
+    spec:
+      containers:
+        - name: nginx
+          image: nginx:1.20.2
+          ports:
+            - containerPort: 80
+```
+
+Apply the following command to creat above example deployment:
+```bash
+kubectl apply -f ./example-manifest/exampledeploy.yaml
+```
+
+Now check the pods, you should see the Kyverno Envoy Sidecar injected:
+
+```bash
+ ~$ kubectl get pods 
+```
+
+Two pods are runing
+```console 
+~$ kubectl get pods 
+NAME                     READY   STATUS    RESTARTS   AGE
+nginx-77b746455b-xntjn   2/2     Running   0          3m46s
+```
+
+3. Check the logs of the Kyverno-envoy-sidecar container to verify the sidecar is running:
+
+```bash
+sanskar@sanskar-HP-Laptop-15s-du1xxx:~$ kubectl logs -n kyverno-envoy-sidecar-injector kyverno-envoy-sidecar-976c94445-nf777 -f 
+time="2024-04-20T13:59:10Z" level=info msg="SimpleServer starting to listen in port 8443"
+time="2024-04-20T14:03:32Z" level=info msg="AdmissionReview for Kind=/v1, Kind=Pod, Namespace=default Name= UID=a57c5c0b-96c0-4c9c-b903-6aa75f635c17 patchOperation=CREATE UserInfo={system:serviceaccount:kube-system:replicaset-controller 9e6576d2-f5c3-4b44-9b9d-952a20b70da7 [system:serviceaccounts system:serviceaccounts:kube-system system:authenticated] map[]}"
+time="2024-04-20T14:03:32Z" level=info msg="sideCar injection for kyverno-envoy-sidecar-injector/nginx-77b746455b-: sidecars: kyverno-envoy-sidecar"
+
+```
diff --git a/sidecar-injector/example-manifest/exampledeploy.yaml b/sidecar-injector/example-manifest/exampledeploy.yaml
@@ -0,0 +1,24 @@
+apiVersion: apps/v1
+kind: Deployment
+metadata:
+  name: nginx
+  namespace: default
+  labels:
+    app.kubernetes.io/name: nginx
+spec:
+  replicas: 1
+  selector:
+    matchLabels:
+      app.kubernetes.io/name: nginx
+  template:
+    metadata:
+      labels:
+        kyverno-envoy-sidecar/injection: enabled
+        app.kubernetes.io/name: nginx
+    spec:
+      containers:
+        - name: nginx
+          image: nginx:1.20.2
+          ports:
+            - containerPort: 80
+
diff --git a/sidecar-injector/example-manifest/policyfile-configmap.yaml b/sidecar-injector/example-manifest/policyfile-configmap.yaml
@@ -0,0 +1,27 @@
+apiVersion: v1
+kind: ConfigMap
+metadata:
+  name: policy-files
+  namespace: default
+data:
+  policy.yaml: |
+    apiVersion: json.kyverno.io/v1alpha1
+    kind: ValidatingPolicy
+    metadata:
+      name: check-dockerfile
+    spec:
+      rules:
+        - name: deny-external-calls
+          assert:
+            all:
+            - message: "HTTP calls are not allowed"
+              check:
+                request:
+                    http:
+                        method: GET
+                        headers:
+                            authorization:
+                                (base64_decode(split(@, ' ')[1])): 
+                                    (split(@, ':')[0]): alice
+                        path: /foo        
+                     
diff --git a/sidecar-injector/go.mod b/sidecar-injector/go.mod
@@ -0,0 +1,53 @@
+module github.com/kyverno/kyverno-envoy-plugin/sidecar-injector
+
+go 1.22.1
+
+require (
+	github.com/ghodss/yaml v1.0.0
+	github.com/pkg/errors v0.9.1
+	github.com/sirupsen/logrus v1.9.3
+	github.com/spf13/cobra v1.8.0
+	k8s.io/api v0.29.3
+	k8s.io/apimachinery v0.29.3
+	k8s.io/client-go v0.29.3
+)
+
+require (
+	github.com/davecgh/go-spew v1.1.1 // indirect
+	github.com/emicklei/go-restful/v3 v3.11.0 // indirect
+	github.com/go-logr/logr v1.3.0 // indirect
+	github.com/go-openapi/jsonpointer v0.19.6 // indirect
+	github.com/go-openapi/jsonreference v0.20.2 // indirect
+	github.com/go-openapi/swag v0.22.3 // indirect
+	github.com/gogo/protobuf v1.3.2 // indirect
+	github.com/golang/protobuf v1.5.4 // indirect
+	github.com/google/gnostic-models v0.6.8 // indirect
+	github.com/google/gofuzz v1.2.0 // indirect
+	github.com/google/uuid v1.3.0 // indirect
+	github.com/imdario/mergo v0.3.6 // indirect
+	github.com/inconshreveable/mousetrap v1.1.0 // indirect
+	github.com/josharian/intern v1.0.0 // indirect
+	github.com/json-iterator/go v1.1.12 // indirect
+	github.com/mailru/easyjson v0.7.7 // indirect
+	github.com/modern-go/concurrent v0.0.0-20180306012644-bacd9c7ef1dd // indirect
+	github.com/modern-go/reflect2 v1.0.2 // indirect
+	github.com/munnerz/goautoneg v0.0.0-20191010083416-a7dc8b61c822 // indirect
+	github.com/spf13/pflag v1.0.5 // indirect
+	golang.org/x/net v0.19.0 // indirect
+	golang.org/x/oauth2 v0.10.0 // indirect
+	golang.org/x/sys v0.15.0 // indirect
+	golang.org/x/term v0.15.0 // indirect
+	golang.org/x/text v0.14.0 // indirect
+	golang.org/x/time v0.3.0 // indirect
+	google.golang.org/appengine v1.6.7 // indirect
+	google.golang.org/protobuf v1.33.0 // indirect
+	gopkg.in/inf.v0 v0.9.1 // indirect
+	gopkg.in/yaml.v2 v2.4.0 // indirect
+	gopkg.in/yaml.v3 v3.0.1 // indirect
+	k8s.io/klog/v2 v2.110.1 // indirect
+	k8s.io/kube-openapi v0.0.0-20231010175941-2dd684a91f00 // indirect
+	k8s.io/utils v0.0.0-20230726121419-3b25d923346b // indirect
+	sigs.k8s.io/json v0.0.0-20221116044647-bc3834ca7abd // indirect
+	sigs.k8s.io/structured-merge-diff/v4 v4.4.1 // indirect
+	sigs.k8s.io/yaml v1.3.0 // indirect
+)