Skip to content

Commit

Permalink
Adjust install instructions to work with pre-configured Cilium `k8sSe…
Browse files Browse the repository at this point in the history
…rviceHost` Helm value

The new install instructions assume that the following configuration is
present somewhere in the Project Syn hierarchy:

```
parameters:
  cilium:
    cilium_helm_values:
      k8sServiceHost: api-int.${openshift:baseDomain}
```
  • Loading branch information
simu committed May 13, 2024
1 parent a884a3a commit d4540cf
Show file tree
Hide file tree
Showing 6 changed files with 68 additions and 21 deletions.
Original file line number Diff line number Diff line change
Expand Up @@ -14,8 +14,8 @@ mkdir -p "${INSTALLER_DIR}"
cat > "${INSTALLER_DIR}/install-config.yaml" <<EOF
apiVersion: v1
metadata:
name: ${CLUSTER_ID}
baseDomain: ${BASE_DOMAIN}
name: ${CLUSTER_ID} <1>
baseDomain: ${BASE_DOMAIN} <1>
platform:
none: {}
networking:
Expand All @@ -25,3 +25,5 @@ pullSecret: |
sshKey: "$(cat $SSH_PUBLIC_KEY)"
EOF
----
<1> Make sure that the values here match the value of `$CLUSTER_DOMAIN` when combined as `<metadata.name>.<baseDomain>`.
Otherwise, the installation will most likely fail.
20 changes: 11 additions & 9 deletions docs/modules/ROOT/partials/install/install-config-openstack.adoc
Original file line number Diff line number Diff line change
Expand Up @@ -24,9 +24,9 @@ EOF
cat > "${INSTALLER_DIR}/install-config.yaml" <<EOF
apiVersion: v1
metadata:
name: ${CLUSTER_ID}
baseDomain: ${BASE_DOMAIN}
compute: <1>
name: ${CLUSTER_ID} <1>
baseDomain: ${BASE_DOMAIN} <1>
compute: <2>
- architecture: amd64
hyperthreading: Enabled
name: worker
Expand All @@ -37,7 +37,7 @@ compute: <1>
rootVolume:
size: 100
type: __DEFAULT__ # TODO: is this generally applicable?
additionalSecurityGroupIDs: <2>
additionalSecurityGroupIDs: <3>
- ${CILIUM_SECURITY_GROUP_ID}
controlPlane:
architecture: amd64
Expand All @@ -50,11 +50,11 @@ controlPlane:
rootVolume:
size: 100
type: __DEFAULT__ # TODO: is this generally applicable?
additionalSecurityGroupIDs: <2>
additionalSecurityGroupIDs: <3>
- ${CILIUM_SECURITY_GROUP_ID}
platform:
openstack:
cloud: shiftstack <3>
cloud: shiftstack <4>
externalNetwork: ${EXTERNAL_NETWORK_NAME}
apiFloatingIP: ${API_VIP}
ingressFloatingIP: ${INGRESS_VIP}
Expand All @@ -67,9 +67,11 @@ pullSecret: |
sshKey: "$(cat $SSH_PUBLIC_KEY)"
EOF
----
<1> We only provision a single compute machine set.
<1> Make sure that the values here match the value of `$CLUSTER_DOMAIN` when combined as `<metadata.name>.<baseDomain>`.
Otherwise, the installation will most likely fail.
<2> We only provision a single compute machine set.
The final machine sets will be configured through Project Syn.
<2> We attach the Cilium security group to both the control plane and the worker nodes.
<3> We attach the Cilium security group to both the control plane and the worker nodes.
This ensures that there's no issues with Cilium traffic during bootstrapping.
<3> This field must match the entry in `clouds` in the `clouds.yaml` file.
<4> This field must match the entry in `clouds` in the `clouds.yaml` file.
If you're following this guide, you shouldn't need to adjust this.
10 changes: 6 additions & 4 deletions docs/modules/ROOT/partials/install/install-config-vsphere.adoc
Original file line number Diff line number Diff line change
Expand Up @@ -12,9 +12,9 @@ mkdir -p "${INSTALLER_DIR}"
cat > "${INSTALLER_DIR}/install-config.yaml" <<EOF
apiVersion: v1
metadata:
name: ${CLUSTER_ID}
baseDomain: ${BASE_DOMAIN}
compute: <1>
name: ${CLUSTER_ID} <1>
baseDomain: ${BASE_DOMAIN} <1>
compute: <2>
- architecture: amd64
hyperthreading: Enabled
name: worker
Expand Down Expand Up @@ -60,5 +60,7 @@ pullSecret: |
sshKey: "$(cat $SSH_PUBLIC_KEY)"
EOF
----
<1> We only provision a single compute machine set.
<1> Make sure that the values here match the value of `$CLUSTER_DOMAIN` when combined as `<metadata.name>.<baseDomain>`.
Otherwise, the installation will most likely fail.
<2> We only provision a single compute machine set.
The final machine sets will be configured through Project Syn.
41 changes: 40 additions & 1 deletion docs/modules/ROOT/partials/install/prepare-commodore.adoc
Original file line number Diff line number Diff line change
Expand Up @@ -18,12 +18,51 @@ mkdir -p inventory/classes/
git clone $(curl -sH"Authorization: Bearer $(commodore fetch-token)" "${COMMODORE_API_URL}/tenants/${TENANT_ID}" | jq -r '.gitRepo.url') inventory/classes/${TENANT_ID}
----

. Add Cilium to cluster configuration
. Configure the cluster's domain in Project Syn
+
[source,bash]
----
export CLUSTER_DOMAIN="${CLUSTER_ID}.${BASE_DOMAIN}" <1>
----
<1> Adjust this as necessary if you're using a non-standard cluster domain.
+
[IMPORTANT]
====
The cluster domain configured here must be correct.
The value is used to configure how Cilium connects to the cluster's K8s API.
====
+
[source,bash]
----
pushd "inventory/classes/${TENANT_ID}/"
yq eval -i ".parameters.openshift.baseDomain = \"${CLUSTER_DOMAIN}\"" \
${CLUSTER_ID}.yml
git commit -a -m "Configure cluster domain for ${CLUSTER_ID}"
----

. Add Cilium to cluster configuration
+
[NOTE]
====
These instructions assume that Cilium is configured to use `api-int.${CLUSTER_DOMAIN}:6443` to connect to the cluster's K8s API.
To ensure that that's the case, add the configuration shown below somewhere in the Project Syn config hierarchy.

[source,yaml]
----
parameters:
cilium:
cilium_helm_values:
k8sServiceHost: api-int.${openshift:baseDomain}
k8sServicePort: "6443"
----

For VSHN, this configuration is set in the https://git.vshn.net/syn/commodore-defaults/-/merge_requests/1789[Commodore global defaults (internal)].
====
+
[source,bash]
----
yq eval -i '.applications += ["cilium"]' ${CLUSTER_ID}.yml
yq eval -i '.parameters.networkpolicy.networkPlugin = "cilium"' ${CLUSTER_ID}.yml
Expand Down
3 changes: 0 additions & 3 deletions docs/modules/ROOT/partials/install/prepare-syn-config.adoc
Original file line number Diff line number Diff line change
Expand Up @@ -18,9 +18,6 @@ fi
+
[source,bash,subs="attributes+"]
----
yq eval -i ".parameters.openshift.baseDomain = \"${CLUSTER_DOMAIN}\"" \
${CLUSTER_ID}.yml

yq eval -i ".parameters.openshift.infraID = \"$(jq -r .infraID "${INSTALLER_DIR}/metadata.json")\"" \
${CLUSTER_ID}.yml

Expand Down
9 changes: 7 additions & 2 deletions docs/modules/ROOT/partials/install/run-installer.adoc
Original file line number Diff line number Diff line change
Expand Up @@ -25,12 +25,17 @@ yq w -i "${INSTALLER_DIR}/manifests/cluster-ingress-02-config.yml" \
cp catalog/manifests/cilium/olm/* target/manifests/
----

. Extract the cluster domain from the generated manifests
. Verify that the generated cluster domain matches the desired cluster domain
+
[source,bash]
----
export CLUSTER_DOMAIN=$(yq e '.spec.baseDomain' \
GEN_CLUSTER_DOMAIN=$(yq e '.spec.baseDomain' \
"${INSTALLER_DIR}/manifests/cluster-dns-02-config.yml")
if [ "$GEN_CLUSTER_DOMAIN" != "$CLUSTER_DOMAIN" ]; then
echo -e "\033[0;31mGenerated cluster domain doesn't match expected cluster domain: Got '$GEN_CLUSTER_DOMAIN', want '$CLUSTER_DOMAIN'\033[0;0m"
else
echo -e "\033[0;32mGenerated cluster domain matches expected cluster domain.\033[0;0m"
fi
----

. Prepare install manifests and ignition config
Expand Down

0 comments on commit d4540cf

Please sign in to comment.