From 9c97e5a1a8a6241923a8a47f4de87b03212e5b8d Mon Sep 17 00:00:00 2001
From: =?UTF-8?q?Marko=20Luk=C5=A1a?= <mluksa@redhat.com>
Date: Tue, 26 Nov 2024 16:33:54 +0100
Subject: [PATCH 1/2] Add documentation for running multiple meshes on a single
 cluster
MIME-Version: 1.0
Content-Type: text/plain; charset=UTF-8
Content-Transfer-Encoding: 8bit

Signed-off-by: Marko Lukša <mluksa@redhat.com>
---
 docs/README.md | 252 +++++++++++++++++++++++++++++++++++++++++++++++++
 1 file changed, 252 insertions(+)

diff --git a/docs/README.md b/docs/README.md
index 136954d3c..b1fa981a8 100644
--- a/docs/README.md
+++ b/docs/README.md
@@ -20,6 +20,14 @@
     - [Example using the InPlace strategy](#example-using-the-inplace-strategy)
   - [RevisionBased](#revisionbased)
     - [Example using the RevisionBased strategy](#example-using-the-revisionbased-strategy)
+- [Multiple meshes on a single cluster](#multiple-meshes-on-a-single-cluster)
+  - [Prerequisites](#prerequisites)
+  - [Installation Steps](#installation-steps)
+    - [Deploying the control planes](#deploying-the-control-planes)
+    - [Deploying the applications](#deploying-the-applications)
+  - [Validation](#validation)
+    - [Checking application to control plane mapping](#checking-application-to-control-plane-mapping)
+    - [Checking application connectivity](#checking-application-connectivity)
 - [Multi-cluster](#multi-cluster)
   - [Prerequisites](#prerequisites)
   - [Common Setup](#common-setup)
@@ -468,6 +476,250 @@ Steps:
     ```
     The old `IstioRevision` resource and the old control plane will be deleted when the grace period specified in the `Istio` resource field `spec.updateStrategy.inactiveRevisionDeletionGracePeriodSeconds` expires.
 
+## Multiple meshes on a single cluster
+
+The Sail Operator supports running multiple meshes on a single cluster and associating each workload with a specific mesh. 
+Each mesh is managed by a separate control plane.
+
+Applications are installed in multiple namespaces, and each namespace is associated with one of the control planes through its labels.
+The `istio.io/rev` label determines which control plane injects the sidecar proxy into the application pods.
+Additional namespace labels determine whether the control plane discovers and manages the resources in the namespace. 
+A control plane will discover and manage only those namespaces that match the discovery selectors configured on the control plane.
+Additionally, discovery selectors determine which control plane creates the `istio-ca-root-cert` ConfigMap in which namespace.
+
+Currently, discovery selectors in multiple control planes must be configured so that they don't overlap (i.e. the discovery selectors of two control planes don't match the same namespace).
+Each control plane must be deployed in a separate Kubernetes namespace.
+
+This guide explains how to set up two meshes: `mesh1` and `mesh2` in namespaces `istio-system1` and `istio-system2`, respectively, and three application namespaces: `app1`, `app2a`, and `app2b`.
+Mesh 1 will manage namespace `app1`, and Mesh 2 will manage namespaces `app2a` and `app2b`.
+Because each mesh will use its own root certificate authority and configured to use a peer authentication policy with the `STRICT` mTLS mode, the communication between the two meshes will not be allowed. 
+
+### Prerequisites
+
+- Install [istioctl](common/install-istioctl-tool.md).
+- Kubernetes 1.23 cluster.
+- kubeconfig file with a context for the Kubernetes cluster.
+- Install the Sail Operator and the Sail CRDs to the cluster.
+
+### Installation Steps
+
+#### Deploying the control planes
+
+1. Create the system namespace `istio-system1` and deploy the `mesh1` control plane in it.
+   ```sh
+   $ kubectl create namespace istio-system1
+   $ kubectl label ns istio-system1 mesh=mesh1
+   $ kubectl apply -f - <<EOF
+   apiVersion: sailoperator.io/v1alpha1
+   kind: Istio
+   metadata:
+     name: mesh1
+   spec:
+     namespace: istio-system1
+     version: v1.24.0
+     values:
+       meshConfig:
+         discoverySelectors:
+         - matchLabels:
+             mesh: mesh1
+   EOF
+   ```
+   
+2. Create the system namespace `istio-system2` and deploy the `mesh2` control plane in it.
+   ```sh
+   $ kubectl create namespace istio-system2
+   $ kubectl label ns istio-system2 mesh=mesh2
+   $ kubectl apply -f - <<EOF
+   apiVersion: sailoperator.io/v1alpha1
+   kind: Istio
+   metadata:
+     name: mesh2
+   spec:
+     namespace: istio-system2
+     version: v1.24.0
+     values:
+       meshConfig:
+         discoverySelectors:
+         - matchLabels:
+             mesh: mesh2
+   EOF
+   ```
+
+3. Create a peer authentication policy that only allows mTLS communication within each mesh.
+   ```sh
+   $ kubectl apply -f - <<EOF
+   apiVersion: security.istio.io/v1
+   kind: PeerAuthentication
+   metadata:
+     name: default
+     namespace: istio-system1
+   spec:
+     mtls:
+       mode: STRICT
+   EOF
+   
+   $ kubectl apply -f - <<EOF
+   apiVersion: security.istio.io/v1beta1
+   kind: PeerAuthentication
+   metadata:
+     name: default
+     namespace: istio-system2
+   spec:
+     mtls:
+       mode: STRICT
+   EOF
+   ```  
+
+#### Verifying the control planes
+
+1. Check the labels on the control plane namespaces:
+   ```sh
+   $ kubectl get ns -l mesh -L mesh
+   NAME            STATUS   AGE    MESH
+   istio-system1   Active   106s   mesh1
+   istio-system2   Active   105s   mesh2
+   ```
+
+2. Check the control planes are `Healthy`:
+   ```sh
+   $ kubectl get istios
+   NAME    REVISIONS   READY   IN USE   ACTIVE REVISION   STATUS    VERSION   AGE
+   mesh1   1           1       0        mesh1             Healthy   v1.24.0   84s
+   mesh2   1           1       0        mesh2             Healthy   v1.24.0   77s
+   ```
+
+3. Confirm that the validation and mutation webhook configurations exist for both meshes:
+   ```sh
+   $ kubectl get validatingwebhookconfigurations
+   NAME                                  WEBHOOKS   AGE
+   istio-validator-mesh1-istio-system1   1          2m45s
+   istio-validator-mesh2-istio-system2   1          2m38s
+
+   $ kubectl get mutatingwebhookconfigurations
+   NAME                                         WEBHOOKS   AGE
+   istio-sidecar-injector-mesh1-istio-system1   2          5m55s
+   istio-sidecar-injector-mesh2-istio-system2   2          5m48s
+   ```
+
+#### Deploying the applications
+
+1. Create three application namespaces:
+   ```sh
+   $ kubectl create ns app1 
+   $ kubectl create ns app2a 
+   $ kubectl create ns app2b
+   ```
+
+2. Label each namespace to enable discovery by the corresponding control plane:
+   ```sh
+   $ kubectl label ns app1 mesh=mesh1
+   $ kubectl label ns app2a mesh=mesh2
+   $ kubectl label ns app2b mesh=mesh2
+   ```
+
+3. Label each namespace to enable injection by the corresponding control plane:
+   ```sh
+   $ kubectl label ns app1 istio.io/rev=mesh1
+   $ kubectl label ns app2a istio.io/rev=mesh2
+   $ kubectl label ns app2b istio.io/rev=mesh2
+   ```
+
+4. Deploy the `curl` and `httpbin` sample applications in each namespace:
+   ```sh
+   $ kubectl -n app1 apply -f https://raw.githubusercontent.com/istio/istio/refs/heads/master/samples/curl/curl.yaml 
+   $ kubectl -n app1 apply -f https://raw.githubusercontent.com/istio/istio/refs/heads/master/samples/httpbin/httpbin.yaml 
+
+   $ kubectl -n app2a apply -f https://raw.githubusercontent.com/istio/istio/refs/heads/master/samples/curl/curl.yaml 
+   $ kubectl -n app2a apply -f https://raw.githubusercontent.com/istio/istio/refs/heads/master/samples/httpbin/httpbin.yaml 
+   
+   $ kubectl -n app2b apply -f https://raw.githubusercontent.com/istio/istio/refs/heads/master/samples/curl/curl.yaml 
+   $ kubectl -n app2b apply -f https://raw.githubusercontent.com/istio/istio/refs/heads/master/samples/httpbin/httpbin.yaml 
+   ```
+
+5. Confirm that a sidecar has been injected into each of the application pods. The value `2/2` should be displayed in the `READY` column for each pod, as in the following example:
+   ```sh
+   $ kubectl get pods -n app1
+   NAME                       READY   STATUS    RESTARTS   AGE
+   curl-5b549b49b8-mg7nl      2/2     Running   0          102s
+   httpbin-7b549f7859-h6hnk   2/2     Running   0          89s
+
+   $ kubectl get pods -n app2a
+   NAME                       READY   STATUS    RESTARTS   AGE
+   curl-5b549b49b8-2hlvm      2/2     Running   0          2m3s
+   httpbin-7b549f7859-bgblg   2/2     Running   0          110s
+
+   $ kubectl get pods -n app2b
+   NAME                       READY   STATUS    RESTARTS   AGE
+   curl-5b549b49b8-xnzzk      2/2     Running   0          2m9s
+   httpbin-7b549f7859-7k5gf   2/2     Running   0          118s
+   ```
+
+### Validation
+
+#### Checking application to control plane mapping
+
+Use the `istioctl ps` command to confirm that the application pods are connected to the correct control plane. 
+
+The `curl` and `httpbin` pods in namespace `app1` should be connected to the control plane in namespace `istio-system1`, as shown in the following example (note the `.app1` suffix in the `NAME` column):
+
+```sh
+$ istioctl ps -i istio-system1
+NAME                              CLUSTER        CDS                LDS                EDS                RDS                ECDS        ISTIOD                            VERSION
+curl-5b549b49b8-mg7nl.app1        Kubernetes     SYNCED (4m40s)     SYNCED (4m40s)     SYNCED (4m31s)     SYNCED (4m40s)     IGNORED     istiod-mesh1-5df45b97dd-tf2wl     1.24.0
+httpbin-7b549f7859-h6hnk.app1     Kubernetes     SYNCED (4m31s)     SYNCED (4m31s)     SYNCED (4m31s)     SYNCED (4m31s)     IGNORED     istiod-mesh1-5df45b97dd-tf2wl     1.24.0
+```
+
+The pods in namespaces `app2a` and `app2b` should be connected to the control plane in namespace `istio-system2`:
+
+```sh
+$ istioctl ps -i istio-system2
+NAME                               CLUSTER        CDS                LDS                EDS                RDS                ECDS        ISTIOD                            VERSION
+curl-5b549b49b8-2hlvm.app2a        Kubernetes     SYNCED (4m37s)     SYNCED (4m37s)     SYNCED (4m31s)     SYNCED (4m37s)     IGNORED     istiod-mesh2-59f6b874fb-mzxqw     1.24.0
+curl-5b549b49b8-xnzzk.app2b        Kubernetes     SYNCED (4m37s)     SYNCED (4m37s)     SYNCED (4m31s)     SYNCED (4m37s)     IGNORED     istiod-mesh2-59f6b874fb-mzxqw     1.24.0
+httpbin-7b549f7859-7k5gf.app2b     Kubernetes     SYNCED (4m31s)     SYNCED (4m31s)     SYNCED (4m31s)     SYNCED (4m31s)     IGNORED     istiod-mesh2-59f6b874fb-mzxqw     1.24.0
+httpbin-7b549f7859-bgblg.app2a     Kubernetes     SYNCED (4m32s)     SYNCED (4m32s)     SYNCED (4m31s)     SYNCED (4m32s)     IGNORED     istiod-mesh2-59f6b874fb-mzxqw     1.24.0
+```
+
+#### Checking application connectivity
+
+As both meshes are configured to use the `STRICT` mTLS peer authentication mode, the applications in namespace `app1` should not be able to communicate with the applications in namespaces `app2a` and `app2b`, and vice versa.
+To test whether the `curl` pod in namespace `app2a` can connect to the `httpbin` service in namespace `app1`, run the following commands:
+
+```sh
+$ kubectl -n app2a exec deploy/curl -c curl -- curl -sIL http://httpbin.app1:8000
+HTTP/1.1 503 Service Unavailable
+content-length: 95
+content-type: text/plain
+date: Fri, 29 Nov 2024 08:58:28 GMT
+server: envoy
+```
+
+As expected, the response indicates that the connection was not successful. 
+In contrast, the same pod should be able to connect to the `httpbin` service in namespace `app2b`, because they are part of the same mesh:
+
+```sh
+$ kubectl -n app2a exec deploy/curl -c curl -- curl -sIL http://httpbin.app2b:8000
+HTTP/1.1 200 OK
+access-control-allow-credentials: true
+access-control-allow-origin: *
+content-security-policy: default-src 'self'; style-src 'self' 'unsafe-inline'; img-src 'self' camo.githubusercontent.com
+content-type: text/html; charset=utf-8
+date: Fri, 29 Nov 2024 08:57:52 GMT
+x-envoy-upstream-service-time: 0
+server: envoy
+transfer-encoding: chunked
+```
+
+### Cleanup
+
+To clean up the resources created in this guide, delete the `Istio` resources and the namespaces:
+
+```sh
+$ kubectl delete istio mesh1 mesh2
+$ kubectl delete ns istio-system1 istio-system2 app1 app2a app2b
+```
+
+
 ## Multi-cluster
 
 You can use the Sail Operator and the Sail CRDs to manage a multi-cluster Istio deployment. The following instructions are adapted from the [Istio multi-cluster documentation](https://istio.io/latest/docs/setup/install/multicluster/) to demonstrate how you can setup the various deployment models with Sail. Please familiarize yourself with the different [deployment models](https://istio.io/latest/docs/ops/deployment/deployment-models/) before starting.

From 9ae72acb5a6981c164d1a7a53f968856541f7964 Mon Sep 17 00:00:00 2001
From: =?UTF-8?q?Marko=20Luk=C5=A1a?= <mluksa@redhat.com>
Date: Tue, 3 Dec 2024 11:20:34 +0100
Subject: [PATCH 2/2] Fix apiVersion
MIME-Version: 1.0
Content-Type: text/plain; charset=UTF-8
Content-Transfer-Encoding: 8bit

Signed-off-by: Marko Lukša <mluksa@redhat.com>
---
 docs/README.md | 2 +-
 1 file changed, 1 insertion(+), 1 deletion(-)

diff --git a/docs/README.md b/docs/README.md
index b1fa981a8..e879a7dc6 100644
--- a/docs/README.md
+++ b/docs/README.md
@@ -559,7 +559,7 @@ Because each mesh will use its own root certificate authority and configured to
    EOF
    
    $ kubectl apply -f - <<EOF
-   apiVersion: security.istio.io/v1beta1
+   apiVersion: security.istio.io/v1
    kind: PeerAuthentication
    metadata:
      name: default