diff --git a/docs/modules/ROOT/pages/how-tos/network/cilium-egress-ip.adoc b/docs/modules/ROOT/pages/how-tos/network/cilium-egress-ip.adoc new file mode 100644 index 00000000..2f359b61 --- /dev/null +++ b/docs/modules/ROOT/pages/how-tos/network/cilium-egress-ip.adoc @@ -0,0 +1,156 @@ += Configure static Egress IPs with Cilium + +== Prerequisites + +* `cluster-admin` privileges +* Cluster is running Cilium. +See the xref:how-tos/network/migrate-to-cilium.adoc[migrating to Cilium] how-to for a guide to migrate a cluster to Cilium. +* `kubectl` +* `jq` +* `curl` +* Working `commodore` command + +== Prepare for configuration + +. Select cluster ++ +[source,bash] +---- +export CLUSTER_ID=c-cluster-id-1234 <1> +export COMMODORE_API_URL=https://api.syn.vshn.net <2> +export TENANT_ID=$(curl -sH "Authorization: Bearer $(commodore fetch-token)" \ + "${COMMODORE_API_URL}/clusters/${CLUSTER_ID}" | jq -r '.tenant') +---- +<1> Replace with the Project Syn cluster ID of the cluster to migrate +<2> Replace with the Lieutenant API on which the cluster is registered + +== Configure Cilium to support static egress IPs + +. Compile cluster catalog ++ +[source,bash] +---- +commodore catalog compile "$CLUSTER_ID" <1> +---- +<1> We recommend switching to an empty directory to run this command. +Alternatively, switch to your existing directory for the cluster. + +. Configure Cilium ++ +[source,bash] +---- +pushd "inventory/classes/$TENANT_ID" +yq -i '.parameters.cilium.cilium_helm_values.egressGateway.enabled=true' \ + "${CLUSTER_ID}.yml" +yq -i '.parameters.cilium.cilium_helm_values.bpf.masquerade=true' \ + "${CLUSTER_ID}.yml" +yq -i '.parameters.cilium.cilium_helm_values.l7Proxy=false' \ + "${CLUSTER_ID}.yml" <1> +---- +<1> Currently (as of Cilium 1.14), the egress gateway feature isn't fully compatible with L7 policies. +To avoid issues, Cilium recommends disabling the L7 proxy when using egress gateway. +See also https://github.com/cilium/cilium/issues/19642[GitHub issue cilium/cilium#19642]. + +. Commit and push changes ++ +[source,bash] +---- +git commit -am "Configure Cilium egress gateway on $CLUSTER_ID" +git push origin master <1> +---- +<1> Optionally, you can push to a branch and merge once you're satisfied with the change. + +. Compile catalog ++ +[source,bash] +---- +popd +commodore catalog compile "$CLUSTER_ID" --push -i +---- + +== Configure egress IPs + +.Exoscale +[%collapsible] +==== +On Exoscale, you can allocate https://community.exoscale.com/documentation/compute/eip/[elastic IPs (EIPs)] to use as static egress IPs. +To use an EIP as a static egress IP, you need to attach it to a cluster node. + +Once you've done that, you can configure the following `CiliumEgresGatewayPolicy`: + +[source,yaml] +---- +apiVersion: cilium.io/v2 +kind: CiliumEgressGatewayPolicy +metadata: + name: egress-ip <1> +spec: + destinationCIDRs: + - 0.0.0.0/0 <2> + egressGroups: + - egressIP: 203.0.113.100 <3> + nodeSelector: + matchLabels: + kubernetes.io/hostname: infra-abcd <4> + selectors: <5> + - podSelector: + matchLabels: + io.kubernetes.pod.namespace: example-namespace <6> +---- +<1> Name of the policy, select a descriptive name if possible. +<2> Destination CIDRs which should be routed through the static egress IP. +In-cluster CIDRs are automatically excluded by Cilium. +<3> Exoscale EIP which you've allocated and attached to a cluster node. +<4> Cluster node to which you've attached the Exoscale EIP. +<5> This section allows you to select pods whose traffic should be routed through the static egress IP +Generally, entries are standard pod selectors. +<6> Cilium supports a special label key `io.kubernetes.pod.namespace` which allows selecting all pods in a specific namespace. + +IMPORTANT: If the `egressIP` and `nodeSelector` of the entry in `egressGroups` don't match, egress traffic will be dropped. +==== + +.Cluster in private network +[%collapsible] +==== +For clusters deployed in a private network, you can use the following configuration to use an IP outside the DHCP range as a static egress IP + +[source,yaml] +---- +apiVersion: cilium.io/v2 +kind: CiliumEgressGatewayPolicy +metadata: + name: egress-ip <1> +spec: + destinationCIDRs: + - 0.0.0.0/0 <2> + excludedCIDRs: + - 172.18.200.0/24 <3> + egressGroups: + - egressIP: 172.18.200.30 <4> + nodeSelector: + matchLabels: + node-role.kubernetes.io/infra: "" <5> + selectors: <6> + - podSelector: + matchLabels: + io.kubernetes.pod.namespace: example-namespace <7> +---- +<1> Name of the policy, select a descriptive name if possible. +<2> Destination CIDRs which should be routed through the static egress IP. +In-cluster CIDRs are automatically excluded by Cilium. +<3> We exclude the cluster's private network CIDR from the set of destinations that should be routed through the static egress IP. +While Cilium auto-excludes the cluster node IPs, we most likely want the whole private network CIDR to be excluded from the egress gateway policy. +Omit this configuration if you want traffic to other systems in the cluster's private network CIDR to be routed through the static egress IP. +<4> Select an IP in the cluster's private network CIDR (outside the DHCP range) as the static egress IP. +<5> Select infra nodes as egress gateway nodes. +Cilium will select one of the nodes to route egress traffic through the defined static egress IP. +<6> This section allows you to select pods whose traffic should be routed through the static egress IP +Generally, entries are standard pod selectors. +<7> Cilium supports a special label key `io.kubernetes.pod.namespace` which allows selecting all pods in a specific namespace. + +NOTE: This configuration is untested, please update the documentation once you've tested this. +==== + +== Upstream documentation + +* https://docs.cilium.io/en/stable/network/egress-gateway/[Cilium egress gateway documentation] diff --git a/docs/modules/ROOT/partials/nav.adoc b/docs/modules/ROOT/partials/nav.adoc index 553765f3..7f869db5 100644 --- a/docs/modules/ROOT/partials/nav.adoc +++ b/docs/modules/ROOT/partials/nav.adoc @@ -103,6 +103,7 @@ * Networking ** xref:oc4:ROOT:how-tos/network/migrate-to-cilium.adoc[] +** xref:oc4:ROOT:how-tos/network/cilium-egress-ip.adoc[] * Ingress ** xref:oc4:ROOT:how-tos/ingress/self-signed-ingress-cert.adoc[]