From 52675f95f9f96c5cd0dd2f65ce0393a8c89d7a1c Mon Sep 17 00:00:00 2001 From: Kfir Toledo Date: Mon, 13 Nov 2023 14:58:27 +0200 Subject: [PATCH 1/5] docs: Installation Guide for ClusterLink Create Installation Guide for ClusterLink in local and remote environemnt. Signed-off-by: Kfir Toledo --- README.md | 2 +- docs/Installation.md | 93 ++++++++++++++++++++++++++++++++++++++++++++ 2 files changed, 94 insertions(+), 1 deletion(-) create mode 100644 docs/Installation.md diff --git a/README.md b/README.md index 3728ed42..dcedfcae 100755 --- a/README.md +++ b/README.md @@ -67,7 +67,7 @@ Here are the key steps for setting up your developer environment, making a chang ### How to setup and run ClusterLink ClusterLink can be set up and run on different environments: local environment (Kind), - Bare-metal environment, or cloud environment. + Bare-metal environment, or cloud environment.For more details, refer to the [Installation Guide for ClusterLink](docs/Installation.md). #### Run ClusterLink in local environment (Kind) diff --git a/docs/Installation.md b/docs/Installation.md new file mode 100644 index 00000000..35d8422b --- /dev/null +++ b/docs/Installation.md @@ -0,0 +1,93 @@ +# Installation Guide for ClusterLink +The ClusterLink gateway contains three main components: the control-plane, data-plane, and gwctl (more details can be found [here](../README.md#what-is-clusterlink)). + +ClusterLink can be deployed in a local environment (Kind) or any remote K8s cluster environment (GKE, EKS, IBM-K8S, etc.). + +To deploy the ClusterLink gateway in a K8s cluster, follow these steps: + +1. Create and deploy certificates and the ClusterLink Deployment file. +2. Expose the ClusterLink deployment to a public IP. +3. Optionally, create a central gwctl component to control all gateways. + +Before you begin, please clone and build the project according to the [instructions](../README.md#building-clustelink). +### 1. ClusterLink Deployment and certificates. +In this step, we build the ClusterLink certificates and deployment files. + +1) Create a folder (eg. TEST_DIR) for all deployments and yaml files: + + export PROJECT_DIR=`git rev-parse --show-toplevel` + export TEST_DIR=$PROJECT_DIR/bin/test/ + mkdir -p $TEST_DIR + cd $TEST_DIR + +2) Create Fabric certificates, including the Root CA certificate for all the clusters (This step needs to be done only once per fabric).: + + $PROJECT_DIR/bin/cl-adm create fabric + +3) Create certificates and a deployment file for the cluster (e.g., peer1). This step needs to be performed for every K8s cluster.: + + $PROJECT_DIR/bin/cl-adm create peer --name peer1 + + Note: By default, the images are pulled from ghcr.io/clusterlink-net. To change the container registry, you can use the ```--container-registry``` flag. + +4) Apply ClusterLink deployment: + + kubectl apply -f $TEST_DIR/peer1/k8s.yaml + + +5) Verify that all components (cl-controlplane, cl-dataplane, gwctl) are set up and ready. + + kubectl get pods + +### 2. Expose the ClusterLink deployment to a public IP +Create public IP for accessing the ClusterLink gateway. +For Local environment create a K8s nodeport service: + + kubectl apply -f $PROJECT_DIR/demos/utils/manifests/kind/cl-svc.yaml + export PEER1_IP=`kubectl get nodes -o "jsonpath={.items[0].status.addresses[0].address}"` + +For a remote environment: + +1. First, create a K8s LoadBalancer service: + ``` + kubectl expose deployment cl-dataplane --name=cl-dataplane-load-balancer --port=443 --target-port=443 --type=LoadBalancer + ``` + +2. Retrieve the LoadBalancer IP when it is allocated: + ``` + export PEER1_IP=$(kubectl get svc -l app=cl-dataplane -o jsonpath="{.items[0].status.loadBalancer.ingress[0].ip}") + ``` + +Now, the ClusterLink gateway can be accessed through `$PEER1_IP` at port 443. + +### 3. Create central gwctl (optional) +By default for each K8s cluster a gwctl pod is created that use REST APIs to send control messages to the +ClusterLink Gateway, using the command: + + kubectl exec -i -- gwctl + +To create a single gwctl that controls all ClusterLink gateways, follow these steps: + +1. Install the local control (gwctl): + + ``` + sudo make install + ``` + +2. Initialize the gwctl CLI for the cluster (e.g., peer1): + ``` + gwctl init --id "peer1" --gwIP $PEER1_IP --gwPort 30443 --certca $TEST_DIR/cert.pem --cert $TEST_DIR/peer1/gwctl/cert.pem --key + ``` + +## Additional Setup Modes +### Debug mode +To run ClusterLink component in debug mode, use ```--log-level``` flag when creating the ClusterLink deployment + + $PROJECT_DIR/bin/cl-adm create peer --name peer1 --log-level debug +### Local docker image +To create and run a local image, first build the project's local images: + + make docker-build +Change the container-registry in the peer deployment to use the local image: + + $PROJECT_DIR/bin/cl-adm create peer --name peer1 --container-registry="" From 38c05da86abe7410c67afb2841b9427d0ff4edd3 Mon Sep 17 00:00:00 2001 From: Kfir Toledo Date: Tue, 14 Nov 2023 12:46:58 +0200 Subject: [PATCH 2/5] Fix for Installation ReadMe Signed-off-by: Kfir Toledo --- README.md | 2 +- demos/iperf3/kind/README.md | 20 ++++---- docs/Installation.md | 91 +++++++++++++++++++++---------------- 3 files changed, 64 insertions(+), 49 deletions(-) diff --git a/README.md b/README.md index dcedfcae..de691da5 100755 --- a/README.md +++ b/README.md @@ -67,7 +67,7 @@ Here are the key steps for setting up your developer environment, making a chang ### How to setup and run ClusterLink ClusterLink can be set up and run on different environments: local environment (Kind), - Bare-metal environment, or cloud environment.For more details, refer to the [Installation Guide for ClusterLink](docs/Installation.md). + Bare-metal environment, or cloud environment. For more details, refer to the [Installation Guide for ClusterLink](docs/Installation.md). #### Run ClusterLink in local environment (Kind) diff --git a/demos/iperf3/kind/README.md b/demos/iperf3/kind/README.md index fc971cb3..fe7af602 100644 --- a/demos/iperf3/kind/README.md +++ b/demos/iperf3/kind/README.md @@ -26,9 +26,9 @@ In this step, we build the Kind cluster with the gateway image. Create certificates and deployments files: 1) Create folder for all deployments and yaml files: - export TEST_DIR=$PROJECT_DIR/bin/test/iperf3/ - mkdir -p $TEST_DIR - cd $TEST_DIR + export DEPLOY_DIR=~/clusterlink-deployment + mkdir -p $DEPLOY_DIR + cd $DEPLOY_DIR 2) Create fabric certificates (includes the Root CA certificate fo all the gateways): @@ -51,7 +51,7 @@ Build the first Kind cluster with gateway, and iperf3-client: 2) Deploy ClusterLink gateway deployment: - kubectl apply -f $TEST_DIR/peer1/k8s.yaml + kubectl apply -f $DEPLOY_DIR/peer1/k8s.yaml 3) Expose ClusterLink gateway port using K8s nodeport service: @@ -70,7 +70,7 @@ Build the second Kind cluster with gateway, and iperf3-client: 2) Deploy Clustrelink gateway deployment: - kubectl apply -f $TEST_DIR/peer2/k8s.yaml + kubectl apply -f $DEPLOY_DIR/peer2/k8s.yaml 3) Expose ClusterLink gateway port using K8s nodeport service: @@ -83,7 +83,9 @@ Build the second Kind cluster with gateway, and iperf3-client: Check that container statuses are Running. - kubectl get pods + kubectl rollout status deployment cl-controlplane + kubectl rollout status deployment cl-dataplane + kubectl wait --for=condition=ready pod -l app=gwctl ### Step 3: Start gwctl In this step, we set up the gwctl. @@ -96,17 +98,17 @@ First, Initialize the Gateways IPs: Initialize gwctl CLI for peer1: - gwctl init --id "peer1" --gwIP $PEER1_IP --gwPort 30443 --certca $TEST_DIR/cert.pem --cert $TEST_DIR/peer1/gwctl/cert.pem --key $TEST_DIR/peer1/gwctl/key.pem + gwctl init --id "peer1" --gwIP $PEER1_IP --gwPort 30443 --certca $DEPLOY_DIR/cert.pem --cert $DEPLOY_DIR/peer1/gwctl/cert.pem --key $DEPLOY_DIR/peer1/gwctl/key.pem Initialize gwctl CLI for peer2: - gwctl init --id "peer2" --gwIP $PEER2_IP --gwPort 30443 --certca $TEST_DIR/cert.pem --cert $TEST_DIR/peer2/gwctl/cert.pem --key $TEST_DIR/peer2/gwctl/key.pem + gwctl init --id "peer2" --gwIP $PEER2_IP --gwPort 30443 --certca $DEPLOY_DIR/cert.pem --cert $DEPLOY_DIR/peer2/gwctl/cert.pem --key $DEPLOY_DIR/peer2/gwctl/key.pem Note : Another approach is to use the gwctl inside the kind cluster. In this case, the gwctl is already set up and deployed. (If you are using macOS this is the better approach). In this case, you should replace the gwctl command with: - kubectl exec -i -- gwctl + kubectl exec -i -- gwctl See example in the next step. diff --git a/docs/Installation.md b/docs/Installation.md index 35d8422b..bc8f9279 100644 --- a/docs/Installation.md +++ b/docs/Installation.md @@ -1,93 +1,106 @@ -# Installation Guide for ClusterLink +# Installation guide for ClusterLink The ClusterLink gateway contains three main components: the control-plane, data-plane, and gwctl (more details can be found [here](../README.md#what-is-clusterlink)). -ClusterLink can be deployed in a local environment (Kind) or any remote K8s cluster environment (GKE, EKS, IBM-K8S, etc.). +ClusterLink can be deployed in a local environment (e.g., Kind) or any remote K8s cluster environment (e.g., google GKE, amazon EKS, IBM IKS, etc.). To deploy the ClusterLink gateway in a K8s cluster, follow these steps: -1. Create and deploy certificates and the ClusterLink Deployment file. +1. Create and deploy certificates and the ClusterLink deployment YAML file. 2. Expose the ClusterLink deployment to a public IP. -3. Optionally, create a central gwctl component to control all gateways. +3. Optionally, create a central gwctl component to control one or more gateways. -Before you begin, please clone and build the project according to the [instructions](../README.md#building-clustelink). -### 1. ClusterLink Deployment and certificates. -In this step, we build the ClusterLink certificates and deployment files. +Before you begin, please build the project according to the [instructions](../README.md#building-clustelink). +## 1. ClusterLink deployment and certificates +In this step, we generate the ClusterLink certificates and deployment files. -1) Create a folder (eg. TEST_DIR) for all deployments and yaml files: +1) Create a folder (eg. DEPLOY_DIR) for all deployments and yaml files: export PROJECT_DIR=`git rev-parse --show-toplevel` - export TEST_DIR=$PROJECT_DIR/bin/test/ - mkdir -p $TEST_DIR - cd $TEST_DIR + export DEPLOY_DIR=~/clusterlink-deployment + mkdir -p $DEPLOY_DIR + cd $DEPLOY_DIR -2) Create Fabric certificates, including the Root CA certificate for all the clusters (This step needs to be done only once per fabric).: +2) Create fabric certificate, which is the root CA certificate for all the clusters: $PROJECT_DIR/bin/cl-adm create fabric -3) Create certificates and a deployment file for the cluster (e.g., peer1). This step needs to be performed for every K8s cluster.: + Note: This step needs to be done only once per fabric. +3) Create certificates and a deployment file for the cluster (e.g., peer1): $PROJECT_DIR/bin/cl-adm create peer --name peer1 - Note: By default, the images are pulled from ghcr.io/clusterlink-net. To change the container registry, you can use the ```--container-registry``` flag. + Note: By default, the images are pulled from ghcr.io/clusterlink-net. To change the container registry, you can use the ```--container-registry``` flag. + This step needs to be performed for every K8s cluster. 4) Apply ClusterLink deployment: - kubectl apply -f $TEST_DIR/peer1/k8s.yaml - + kubectl apply -f $DEPLOY_DIR/peer1/k8s.yaml 5) Verify that all components (cl-controlplane, cl-dataplane, gwctl) are set up and ready. - kubectl get pods - -### 2. Expose the ClusterLink deployment to a public IP -Create public IP for accessing the ClusterLink gateway. -For Local environment create a K8s nodeport service: + kubectl rollout status deployment cl-controlplane + kubectl rollout status deployment cl-dataplane + kubectl wait --for=condition=ready pod -l app=gwctl + + Expected output: + + deployment "cl-controlplane" successfully rolled out + deployment "cl-dataplane" successfully rolled out + pod/gwctl condition met + +## 2. Expose the ClusterLink deployment to a public IP +Create a public IP for accessing the ClusterLink gateway. +For a local environment (e.g., Kind), create a K8s nodeport service: kubectl apply -f $PROJECT_DIR/demos/utils/manifests/kind/cl-svc.yaml export PEER1_IP=`kubectl get nodes -o "jsonpath={.items[0].status.addresses[0].address}"` + export PEER1_PORT=30443 -For a remote environment: +For a remote K8s cluster environment (e.g., google GKE, amazon EKS, IBM IKS, etc.): 1. First, create a K8s LoadBalancer service: - ``` + kubectl expose deployment cl-dataplane --name=cl-dataplane-load-balancer --port=443 --target-port=443 --type=LoadBalancer - ``` 2. Retrieve the LoadBalancer IP when it is allocated: - ``` + export PEER1_IP=$(kubectl get svc -l app=cl-dataplane -o jsonpath="{.items[0].status.loadBalancer.ingress[0].ip}") - ``` + export PEER1_PORT=443 -Now, the ClusterLink gateway can be accessed through `$PEER1_IP` at port 443. +Now, the ClusterLink gateway can be accessed through `$PEER1_IP` at port `$PEER1_PORT`. -### 3. Create central gwctl (optional) +## 3. Create a central gwctl (optional) By default for each K8s cluster a gwctl pod is created that use REST APIs to send control messages to the -ClusterLink Gateway, using the command: +ClusterLink gateway, using the command: - kubectl exec -i -- gwctl + kubectl exec -i -- gwctl -To create a single gwctl that controls all ClusterLink gateways, follow these steps: +To create a single gwctl that controls one or more ClusterLink gateways, follow these steps: 1. Install the local control (gwctl): - ``` sudo make install - ``` 2. Initialize the gwctl CLI for the cluster (e.g., peer1): - ``` - gwctl init --id "peer1" --gwIP $PEER1_IP --gwPort 30443 --certca $TEST_DIR/cert.pem --cert $TEST_DIR/peer1/gwctl/cert.pem --key - ``` -## Additional Setup Modes + gwctl init --id peer1 --gwIP $PEER1_IP --gwPort 30443 --certca $DEPLOY_DIR/cert.pem --cert $DEPLOY_DIR/peer1/gwctl/cert.pem --key + +3. To run gwctl command: + + gwctl --myid peer1 + +## Additional setup modes ### Debug mode -To run ClusterLink component in debug mode, use ```--log-level``` flag when creating the ClusterLink deployment +To run ClusterLink components in debug mode, use ```--log-level``` flag when creating the ClusterLink deployment $PROJECT_DIR/bin/cl-adm create peer --name peer1 --log-level debug -### Local docker image + +### Running self-built images To create and run a local image, first build the project's local images: make docker-build + Change the container-registry in the peer deployment to use the local image: $PROJECT_DIR/bin/cl-adm create peer --name peer1 --container-registry="" + From f49e3983b5712d6711f76b4cbf63fe15e4e041bb Mon Sep 17 00:00:00 2001 From: Kfir Toledo Date: Tue, 14 Nov 2023 15:00:37 +0200 Subject: [PATCH 3/5] Rename file to docs/installation.md Signed-off-by: Kfir Toledo --- docs/{Installation.md => installation.md} | 0 1 file changed, 0 insertions(+), 0 deletions(-) rename docs/{Installation.md => installation.md} (100%) diff --git a/docs/Installation.md b/docs/installation.md similarity index 100% rename from docs/Installation.md rename to docs/installation.md From 237baf6405b26c87f3e0a4a45bc168777ce2f786 Mon Sep 17 00:00:00 2001 From: Kfir Toledo Date: Wed, 15 Nov 2023 10:13:22 +0200 Subject: [PATCH 4/5] Remove local and remote phrases Signed-off-by: Kfir Toledo --- docs/installation.md | 6 +++--- 1 file changed, 3 insertions(+), 3 deletions(-) diff --git a/docs/installation.md b/docs/installation.md index bc8f9279..828abc10 100644 --- a/docs/installation.md +++ b/docs/installation.md @@ -1,7 +1,7 @@ # Installation guide for ClusterLink The ClusterLink gateway contains three main components: the control-plane, data-plane, and gwctl (more details can be found [here](../README.md#what-is-clusterlink)). -ClusterLink can be deployed in a local environment (e.g., Kind) or any remote K8s cluster environment (e.g., google GKE, amazon EKS, IBM IKS, etc.). +ClusterLink can be deployed in any K8s based cluster (e.g., google GKE, amazon EKS, IBM IKS, KIND etc.). To deploy the ClusterLink gateway in a K8s cluster, follow these steps: @@ -50,13 +50,13 @@ In this step, we generate the ClusterLink certificates and deployment files. ## 2. Expose the ClusterLink deployment to a public IP Create a public IP for accessing the ClusterLink gateway. -For a local environment (e.g., Kind), create a K8s nodeport service: +For a testing environment (e.g., KIND), create a K8s nodeport service: kubectl apply -f $PROJECT_DIR/demos/utils/manifests/kind/cl-svc.yaml export PEER1_IP=`kubectl get nodes -o "jsonpath={.items[0].status.addresses[0].address}"` export PEER1_PORT=30443 -For a remote K8s cluster environment (e.g., google GKE, amazon EKS, IBM IKS, etc.): +For a operational K8s cluster environment (e.g., google GKE, amazon EKS, IBM IKS, etc.): 1. First, create a K8s LoadBalancer service: From 67d20678724f3dc0e66f7c881c4d1686368ee8ae Mon Sep 17 00:00:00 2001 From: Kfir Toledo Date: Thu, 16 Nov 2023 14:21:05 +0200 Subject: [PATCH 5/5] Add key file and replace nodeport file with an inline command Signed-off-by: Kfir Toledo --- docs/installation.md | 50 +++++++++++++++++++++++++++++++------------- 1 file changed, 35 insertions(+), 15 deletions(-) diff --git a/docs/installation.md b/docs/installation.md index 828abc10..6689627f 100644 --- a/docs/installation.md +++ b/docs/installation.md @@ -1,4 +1,5 @@ # Installation guide for ClusterLink + The ClusterLink gateway contains three main components: the control-plane, data-plane, and gwctl (more details can be found [here](../README.md#what-is-clusterlink)). ClusterLink can be deployed in any K8s based cluster (e.g., google GKE, amazon EKS, IBM IKS, KIND etc.). @@ -10,7 +11,9 @@ To deploy the ClusterLink gateway in a K8s cluster, follow these steps: 3. Optionally, create a central gwctl component to control one or more gateways. Before you begin, please build the project according to the [instructions](../README.md#building-clustelink). + ## 1. ClusterLink deployment and certificates + In this step, we generate the ClusterLink certificates and deployment files. 1) Create a folder (eg. DEPLOY_DIR) for all deployments and yaml files: @@ -49,25 +52,42 @@ In this step, we generate the ClusterLink certificates and deployment files. pod/gwctl condition met ## 2. Expose the ClusterLink deployment to a public IP -Create a public IP for accessing the ClusterLink gateway. -For a testing environment (e.g., KIND), create a K8s nodeport service: - kubectl apply -f $PROJECT_DIR/demos/utils/manifests/kind/cl-svc.yaml - export PEER1_IP=`kubectl get nodes -o "jsonpath={.items[0].status.addresses[0].address}"` - export PEER1_PORT=30443 - -For a operational K8s cluster environment (e.g., google GKE, amazon EKS, IBM IKS, etc.): +Create a public IP for accessing the ClusterLink gateway. +* For a testing environment (e.g., KIND), create a K8s nodeport service: +``` +echo " +apiVersion: v1 +kind: Service +metadata: + name: cl-svc +spec: + type: NodePort + selector: + app: cl-dataplane + ports: + - port: 443 + targetPort: 443 + nodePort: 30443 + protocol: TCP + name: http +" | kubectl apply -f - + +export PEER1_IP=`kubectl get nodes -o "jsonpath={.items[0].status.addresses[0].address}"` +export PEER1_PORT=30443 +``` +* For a operational K8s cluster environment (e.g., google GKE, amazon EKS, IBM IKS, etc.): 1. First, create a K8s LoadBalancer service: - kubectl expose deployment cl-dataplane --name=cl-dataplane-load-balancer --port=443 --target-port=443 --type=LoadBalancer + kubectl expose deployment cl-dataplane --name=cl-dataplane-load-balancer --port=443 --target-port=443 --type=LoadBalancer 2. Retrieve the LoadBalancer IP when it is allocated: - export PEER1_IP=$(kubectl get svc -l app=cl-dataplane -o jsonpath="{.items[0].status.loadBalancer.ingress[0].ip}") - export PEER1_PORT=443 + export PEER1_IP=$(kubectl get svc -l app=cl-dataplane -o jsonpath="{.items[0].status.loadBalancer.ingress[0].ip}") + export PEER1_PORT=443 -Now, the ClusterLink gateway can be accessed through `$PEER1_IP` at port `$PEER1_PORT`. +Now, the ClusterLink gateway can be accessed through `$PEER1_IP` at port `$PEER1_PORT`. ## 3. Create a central gwctl (optional) By default for each K8s cluster a gwctl pod is created that use REST APIs to send control messages to the @@ -79,15 +99,15 @@ To create a single gwctl that controls one or more ClusterLink gateways, follow 1. Install the local control (gwctl): - sudo make install + sudo make install 2. Initialize the gwctl CLI for the cluster (e.g., peer1): - gwctl init --id peer1 --gwIP $PEER1_IP --gwPort 30443 --certca $DEPLOY_DIR/cert.pem --cert $DEPLOY_DIR/peer1/gwctl/cert.pem --key + gwctl init --id peer1 --gwIP $PEER1_IP --gwPort $PEER1_PORT --certca $DEPLOY_DIR/cert.pem --cert $DEPLOY_DIR/peer1/gwctl/cert.pem --key $DEPLOY_DIR/peer1/gwctl/key.pem 3. To run gwctl command: - - gwctl --myid peer1 + + gwctl --myid peer1 ## Additional setup modes ### Debug mode