Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Update Cloudscale install documentation to enable machine-api provider #365

Merged
merged 4 commits into from
Nov 18, 2024
Merged

Update Cloudscale install documentation to enable machine-api provider #365

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
4 commits
Select commit Hold shift + click to select a range
3d74356
Update Cloudscale install documentation to enable machine-api provider
HappyTetrahedron Nov 15, 2024
d79c4db
Update docs/modules/ROOT/partials/install/bootstrap-nodes.adoc
HappyTetrahedron Nov 15, 2024
b263563
Update docs/modules/ROOT/partials/install/bootstrap-nodes.adoc
HappyTetrahedron Nov 15, 2024
86b47ef
Add step to manually update LB hieradata
HappyTetrahedron Nov 18, 2024
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
78 changes: 73 additions & 5 deletions docs/modules/ROOT/partials/install/bootstrap-nodes.adoc
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -98,6 +98,32 @@ terraform apply

ifeval::["{provider}" == "cloudscale"]
. Add the DNS records for etcd shown in output variable `dns_entries` from the previous step to the cluster's parent zone

. Apply the manifests for the cloudscale machine-api provider
+
[source,bash,subs="attributes+"]
----
kapitan refs --reveal --refs-path ../../refs ../machine-api-provider-cloudscale/00_secrets.yaml | kubectl apply -f -

kubectl apply -f ../machine-api-provider-cloudscale/10_clusterRoleBinding.yaml

kubectl apply -f ../machine-api-provider-cloudscale/10_serviceAccount.yaml

kubectl apply -f ../machine-api-provider-cloudscale/11_deployment.yaml
----

. Apply the machinesets from terraform
+
[source,bash,subs="attributes+"]
----
terraform output -raw worker-machineset_yml | grep -vP '^(│|╵|╷|There are some problems with the CLI configuration)' | yq -P > worker-machineset.yml
head worker-machineset.yml
kubectl apply -f worker-machineset.yml

terraform output -raw infra-machineset_yml | grep -vP '^(│|╵|╷|There are some problems with the CLI configuration)' | yq -P > infra-machineset.yml
head infra-machineset.yml
kubectl apply -f infra-machineset.yml
----
endif::[]

. Wait for bootstrap to complete
Expand All @@ -122,7 +148,12 @@ kubectl taint no -l node-role.kubernetes.io/master node.cloudprovider.kubernetes
----
--

ifeval::["{provider}" != "cloudscale"]
. Remove bootstrap node and provision remaining nodes
endif::[]
ifeval::["{provider}" == "cloudscale"]
. Remove bootstrap node
endif::[]
+
[source,bash,subs="attributes+"]
----
Expand All @@ -132,6 +163,30 @@ terraform apply
popd
----

. Review and merge the LB hieradata MR (listed in Terraform output `hieradata_mr`) and run Puppet on the LBs after the deploy job has completed
+
[source,bash]
----
for fqdn in "${LB_FQDNS[@]}"; do
ssh "${fqdn}" sudo puppetctl run
done
----

. Scale up the infra and worker machinesets
+
[source,bash]
----
INFRA_NODES=4 # adjust to desired number of infra nodes
WORKER_NODES=3 # adjust to desired number of worker nodes
----
+
[source,bash]
----
kubectl scale machineset -nopenshift-machine-api infra --replicas="${INFRA_NODES}"
kubectl scale machineset -nopenshift-machine-api worker --replicas="${WORKER_NODES}"
----
endif::[]

. Disable OpenShift kube-proxy deployment and revert Cilium patch
+
[source,bash]
Expand All @@ -146,15 +201,27 @@ done && echo -e "\nCilium config updated"
kubectl -n cilium rollout restart ds/cilium
----

. Review and merge the LB hieradata MR (listed in Terraform output `hieradata_mr`) and run Puppet on the LBs after the deploy job has completed
ifeval::["{provider}" == "cloudscale"]
. Add Infra Node IPs to LB Hieradata
+
[source,bash]
----
for fqdn in "${LB_FQDNS[@]}"; do
ssh "${fqdn}" sudo puppetctl run
done
----
git clone [email protected]:appuio/appuio_hieradata.git
Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The terraform checkout of this repo uses the project bot user via https, and trying to do anything in it asks for the bot's password, which is annoying to script for.
I may be missing something. But I found it easier to just re-clone with ssh...

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

You could maybe do something like

git remote add origin-git [email protected]:appuio/appuio-hieradata.git

in the Terraform checkout and then

git push origin-git master

but that's not easier than just cloning the repo again.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

yeah... plus, in my local directory the terraform-managed checkout wasn't in a clean state. Whether that was a result of me doing things wrong or a normal possibility, I'm not sure. But a fresh clone certainly never has issues with unclean git trees...


pushd appuio_hieradata/lbaas

kubectl get node -l "node-role.kubernetes.io/infra" -oyaml | yq '.items[].status.addresses | filter(.type == "InternalIP") | map(.address)' > ips.yml

yq -i '."profile_openshift4_gateway::backends".router = load("ips.yml")' "${CLUSTER_ID}.yaml"

rm ips.yml

git commit -am "Add infra nodes as backends for ${CLUSTER_ID}."
git push
popd
----
endif::[]
ifeval::["{provider}" != "cloudscale"]
. Approve node certs
+
include::partial$install/approve-node-csrs.adoc[]
Expand Down Expand Up @@ -185,6 +252,7 @@ kubectl get node -ojson | \
+
[NOTE]
At this point you may want to add extra labels to the additional worker groups, if there are any.
endif::[]

. Enable proxy protocol on ingress controller
+
Expand Down
20 changes: 20 additions & 0 deletions docs/modules/ROOT/partials/install/prepare-syn-config-machine-api.adoc
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -0,0 +1,20 @@
ifeval::["{provider}" == "cloudscale"]
. Prepare cloudscale machine-api provider
+
[source,bash,subs="attributes+"]
----
yq eval -i ".parameters.openshift4_terraform.terraform_variables.worker_count = 0" \
${CLUSTER_ID}.yml

yq eval -i ".parameters.openshift4_terraform.terraform_variables.infra_count = 0" \
${CLUSTER_ID}.yml


yq -i '.applications += "machine-api-provider-cloudscale"' \
${CLUSTER_ID}.yml
yq eval -i ".parameters.openshift4_terraform.terraform_variables.make_worker_adoptable_by_provider = true" \
${CLUSTER_ID}.yml
yq eval -i '.parameters.machine_api_provider_cloudscale.secrets["cloudscale-user-data"].stringData.ignitionCA = "${openshift4_terraform:terraform_variables:ignition_ca}"' \
${CLUSTER_ID}.yml
----
endif::[]
4 changes: 4 additions & 0 deletions docs/modules/ROOT/partials/install/prepare-syn-config.adoc
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -62,6 +62,7 @@ yq eval -i ".parameters.openshift4_version.spec.channel = \"fast-{ocp-minor-vers

ifeval::["{provider}" == "cloudscale"]
:is-terraform: yes
:use-machine-api: yes
endif::[]
ifeval::["{provider}" == "exoscale"]
:is-terraform: yes
Expand All @@ -70,6 +71,9 @@ endif::[]
ifeval::["{is-terraform}" == "yes"]
include::partial$install/prepare-syn-config-terraform.adoc[]
endif::[]
ifeval::["{use-machine-api}" == "yes"]
include::partial$install/prepare-syn-config-machine-api.adoc[]
endif::[]
ifeval::["{provider}" == "vsphere"]
include::partial$install/prepare-syn-config-vsphere.adoc[]
endif::[]
Expand Down