Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Add documentation for running multiple meshes on a single cluster #506

Merged
merged 2 commits into from
Dec 3, 2024
Merged

Add documentation for running multiple meshes on a single cluster #506

Changes from all commits
Commits
Show all changes
2 commits
Select commit Hold shift + click to select a range
9c97e5a
Add documentation for running multiple meshes on a single cluster
luksa Nov 26, 2024
9ae72ac
Fix apiVersion
luksa Dec 3, 2024
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
252 changes: 252 additions & 0 deletions docs/README.md
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -20,6 +20,14 @@
- [Example using the InPlace strategy](#example-using-the-inplace-strategy)
- [RevisionBased](#revisionbased)
- [Example using the RevisionBased strategy](#example-using-the-revisionbased-strategy)
- [Multiple meshes on a single cluster](#multiple-meshes-on-a-single-cluster)
- [Prerequisites](#prerequisites)
- [Installation Steps](#installation-steps)
- [Deploying the control planes](#deploying-the-control-planes)
- [Deploying the applications](#deploying-the-applications)
- [Validation](#validation)
- [Checking application to control plane mapping](#checking-application-to-control-plane-mapping)
- [Checking application connectivity](#checking-application-connectivity)
- [Multi-cluster](#multi-cluster)
- [Prerequisites](#prerequisites)
- [Common Setup](#common-setup)
Expand Down Expand Up @@ -468,6 +476,250 @@ Steps:
```
The old `IstioRevision` resource and the old control plane will be deleted when the grace period specified in the `Istio` resource field `spec.updateStrategy.inactiveRevisionDeletionGracePeriodSeconds` expires.

## Multiple meshes on a single cluster

The Sail Operator supports running multiple meshes on a single cluster and associating each workload with a specific mesh.
Each mesh is managed by a separate control plane.

Applications are installed in multiple namespaces, and each namespace is associated with one of the control planes through its labels.
The `istio.io/rev` label determines which control plane injects the sidecar proxy into the application pods.
Additional namespace labels determine whether the control plane discovers and manages the resources in the namespace.
A control plane will discover and manage only those namespaces that match the discovery selectors configured on the control plane.
Additionally, discovery selectors determine which control plane creates the `istio-ca-root-cert` ConfigMap in which namespace.

Currently, discovery selectors in multiple control planes must be configured so that they don't overlap (i.e. the discovery selectors of two control planes don't match the same namespace).
Each control plane must be deployed in a separate Kubernetes namespace.

This guide explains how to set up two meshes: `mesh1` and `mesh2` in namespaces `istio-system1` and `istio-system2`, respectively, and three application namespaces: `app1`, `app2a`, and `app2b`.
Mesh 1 will manage namespace `app1`, and Mesh 2 will manage namespaces `app2a` and `app2b`.
Because each mesh will use its own root certificate authority and configured to use a peer authentication policy with the `STRICT` mTLS mode, the communication between the two meshes will not be allowed.

### Prerequisites

- Install [istioctl](common/install-istioctl-tool.md).
- Kubernetes 1.23 cluster.
- kubeconfig file with a context for the Kubernetes cluster.
- Install the Sail Operator and the Sail CRDs to the cluster.

### Installation Steps

#### Deploying the control planes

1. Create the system namespace `istio-system1` and deploy the `mesh1` control plane in it.
```sh
$ kubectl create namespace istio-system1
$ kubectl label ns istio-system1 mesh=mesh1
$ kubectl apply -f - <<EOF
apiVersion: sailoperator.io/v1alpha1
kind: Istio
metadata:
name: mesh1
spec:
namespace: istio-system1
version: v1.24.0
values:
meshConfig:
discoverySelectors:
- matchLabels:
mesh: mesh1
EOF
```

2. Create the system namespace `istio-system2` and deploy the `mesh2` control plane in it.
```sh
$ kubectl create namespace istio-system2
$ kubectl label ns istio-system2 mesh=mesh2
$ kubectl apply -f - <<EOF
apiVersion: sailoperator.io/v1alpha1
kind: Istio
metadata:
name: mesh2
spec:
namespace: istio-system2
version: v1.24.0
values:
meshConfig:
discoverySelectors:
- matchLabels:
mesh: mesh2
EOF
```

3. Create a peer authentication policy that only allows mTLS communication within each mesh.
```sh
$ kubectl apply -f - <<EOF
apiVersion: security.istio.io/v1
kind: PeerAuthentication
metadata:
name: default
namespace: istio-system1
spec:
mtls:
mode: STRICT
EOF

$ kubectl apply -f - <<EOF
apiVersion: security.istio.io/v1
kind: PeerAuthentication
metadata:
name: default
namespace: istio-system2
spec:
mtls:
mode: STRICT
EOF
```

#### Verifying the control planes

1. Check the labels on the control plane namespaces:
```sh
$ kubectl get ns -l mesh -L mesh
NAME STATUS AGE MESH
istio-system1 Active 106s mesh1
istio-system2 Active 105s mesh2
```

2. Check the control planes are `Healthy`:
```sh
$ kubectl get istios
NAME REVISIONS READY IN USE ACTIVE REVISION STATUS VERSION AGE
mesh1 1 1 0 mesh1 Healthy v1.24.0 84s
mesh2 1 1 0 mesh2 Healthy v1.24.0 77s
```

3. Confirm that the validation and mutation webhook configurations exist for both meshes:
```sh
$ kubectl get validatingwebhookconfigurations
NAME WEBHOOKS AGE
istio-validator-mesh1-istio-system1 1 2m45s
istio-validator-mesh2-istio-system2 1 2m38s

$ kubectl get mutatingwebhookconfigurations
NAME WEBHOOKS AGE
istio-sidecar-injector-mesh1-istio-system1 2 5m55s
istio-sidecar-injector-mesh2-istio-system2 2 5m48s
```

#### Deploying the applications

1. Create three application namespaces:
```sh
$ kubectl create ns app1
$ kubectl create ns app2a
$ kubectl create ns app2b
```

2. Label each namespace to enable discovery by the corresponding control plane:
```sh
$ kubectl label ns app1 mesh=mesh1
$ kubectl label ns app2a mesh=mesh2
$ kubectl label ns app2b mesh=mesh2
```

3. Label each namespace to enable injection by the corresponding control plane:
```sh
$ kubectl label ns app1 istio.io/rev=mesh1
$ kubectl label ns app2a istio.io/rev=mesh2
$ kubectl label ns app2b istio.io/rev=mesh2
```

4. Deploy the `curl` and `httpbin` sample applications in each namespace:
```sh
$ kubectl -n app1 apply -f https://raw.githubusercontent.com/istio/istio/refs/heads/master/samples/curl/curl.yaml
$ kubectl -n app1 apply -f https://raw.githubusercontent.com/istio/istio/refs/heads/master/samples/httpbin/httpbin.yaml

$ kubectl -n app2a apply -f https://raw.githubusercontent.com/istio/istio/refs/heads/master/samples/curl/curl.yaml
$ kubectl -n app2a apply -f https://raw.githubusercontent.com/istio/istio/refs/heads/master/samples/httpbin/httpbin.yaml

$ kubectl -n app2b apply -f https://raw.githubusercontent.com/istio/istio/refs/heads/master/samples/curl/curl.yaml
$ kubectl -n app2b apply -f https://raw.githubusercontent.com/istio/istio/refs/heads/master/samples/httpbin/httpbin.yaml
```

5. Confirm that a sidecar has been injected into each of the application pods. The value `2/2` should be displayed in the `READY` column for each pod, as in the following example:
```sh
$ kubectl get pods -n app1
NAME READY STATUS RESTARTS AGE
curl-5b549b49b8-mg7nl 2/2 Running 0 102s
httpbin-7b549f7859-h6hnk 2/2 Running 0 89s

$ kubectl get pods -n app2a
NAME READY STATUS RESTARTS AGE
curl-5b549b49b8-2hlvm 2/2 Running 0 2m3s
httpbin-7b549f7859-bgblg 2/2 Running 0 110s

$ kubectl get pods -n app2b
NAME READY STATUS RESTARTS AGE
curl-5b549b49b8-xnzzk 2/2 Running 0 2m9s
httpbin-7b549f7859-7k5gf 2/2 Running 0 118s
```

### Validation

#### Checking application to control plane mapping

Use the `istioctl ps` command to confirm that the application pods are connected to the correct control plane.

The `curl` and `httpbin` pods in namespace `app1` should be connected to the control plane in namespace `istio-system1`, as shown in the following example (note the `.app1` suffix in the `NAME` column):

```sh
$ istioctl ps -i istio-system1
NAME CLUSTER CDS LDS EDS RDS ECDS ISTIOD VERSION
curl-5b549b49b8-mg7nl.app1 Kubernetes SYNCED (4m40s) SYNCED (4m40s) SYNCED (4m31s) SYNCED (4m40s) IGNORED istiod-mesh1-5df45b97dd-tf2wl 1.24.0
httpbin-7b549f7859-h6hnk.app1 Kubernetes SYNCED (4m31s) SYNCED (4m31s) SYNCED (4m31s) SYNCED (4m31s) IGNORED istiod-mesh1-5df45b97dd-tf2wl 1.24.0
```

The pods in namespaces `app2a` and `app2b` should be connected to the control plane in namespace `istio-system2`:

```sh
$ istioctl ps -i istio-system2
NAME CLUSTER CDS LDS EDS RDS ECDS ISTIOD VERSION
curl-5b549b49b8-2hlvm.app2a Kubernetes SYNCED (4m37s) SYNCED (4m37s) SYNCED (4m31s) SYNCED (4m37s) IGNORED istiod-mesh2-59f6b874fb-mzxqw 1.24.0
curl-5b549b49b8-xnzzk.app2b Kubernetes SYNCED (4m37s) SYNCED (4m37s) SYNCED (4m31s) SYNCED (4m37s) IGNORED istiod-mesh2-59f6b874fb-mzxqw 1.24.0
httpbin-7b549f7859-7k5gf.app2b Kubernetes SYNCED (4m31s) SYNCED (4m31s) SYNCED (4m31s) SYNCED (4m31s) IGNORED istiod-mesh2-59f6b874fb-mzxqw 1.24.0
httpbin-7b549f7859-bgblg.app2a Kubernetes SYNCED (4m32s) SYNCED (4m32s) SYNCED (4m31s) SYNCED (4m32s) IGNORED istiod-mesh2-59f6b874fb-mzxqw 1.24.0
```

#### Checking application connectivity

As both meshes are configured to use the `STRICT` mTLS peer authentication mode, the applications in namespace `app1` should not be able to communicate with the applications in namespaces `app2a` and `app2b`, and vice versa.
To test whether the `curl` pod in namespace `app2a` can connect to the `httpbin` service in namespace `app1`, run the following commands:

```sh
$ kubectl -n app2a exec deploy/curl -c curl -- curl -sIL http://httpbin.app1:8000
HTTP/1.1 503 Service Unavailable
content-length: 95
content-type: text/plain
date: Fri, 29 Nov 2024 08:58:28 GMT
server: envoy
```

As expected, the response indicates that the connection was not successful.
In contrast, the same pod should be able to connect to the `httpbin` service in namespace `app2b`, because they are part of the same mesh:

```sh
$ kubectl -n app2a exec deploy/curl -c curl -- curl -sIL http://httpbin.app2b:8000
HTTP/1.1 200 OK
access-control-allow-credentials: true
access-control-allow-origin: *
content-security-policy: default-src 'self'; style-src 'self' 'unsafe-inline'; img-src 'self' camo.githubusercontent.com
content-type: text/html; charset=utf-8
date: Fri, 29 Nov 2024 08:57:52 GMT
x-envoy-upstream-service-time: 0
server: envoy
transfer-encoding: chunked
```

### Cleanup

To clean up the resources created in this guide, delete the `Istio` resources and the namespaces:

```sh
$ kubectl delete istio mesh1 mesh2
$ kubectl delete ns istio-system1 istio-system2 app1 app2a app2b
```


## Multi-cluster

You can use the Sail Operator and the Sail CRDs to manage a multi-cluster Istio deployment. The following instructions are adapted from the [Istio multi-cluster documentation](https://istio.io/latest/docs/setup/install/multicluster/) to demonstrate how you can setup the various deployment models with Sail. Please familiarize yourself with the different [deployment models](https://istio.io/latest/docs/ops/deployment/deployment-models/) before starting.
Expand Down
Loading