Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Update decommissioning instructions for cloudscale #374

Merged
merged 1 commit into from
Dec 19, 2024
Merged
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
39 changes: 18 additions & 21 deletions docs/modules/ROOT/pages/how-tos/cloudscale/decommission.adoc
Original file line number Diff line number Diff line change
Expand Up @@ -21,7 +21,6 @@ include::partial$cloudscale/prerequisites.adoc[]
+
[source,bash]
----
export CLUSTER_ID=<lieutenant-cluster-id>
export GITLAB_TOKEN=<gitlab-api-token> # From https://git.vshn.net/-/user_settings/personal_access_tokens
export GITLAB_USER=<gitlab-user-name>
----
Expand All @@ -30,24 +29,17 @@ export GITLAB_USER=<gitlab-user-name>
+
include::partial$connect-to-vault.adoc[]
+
include::partial$commodore-init.adoc[]
+
[source,bash]
----
export TENANT_ID=$(curl -sH "Authorization: Bearer $(commodore fetch-token)" ${COMMODORE_API_URL}/clusters/${CLUSTER_ID} | jq -r .tenant)
export CLOUDSCALE_API_TOKEN=$(vault kv get -format=json clusters/kv/$TENANT_ID/$CLUSTER_ID/cloudscale | jq -r .data.data.token)
export REGION=$(curl -sH "Authorization: Bearer $(commodore fetch-token)" ${COMMODORE_API_URL}/clusters/${CLUSTER_ID} | jq -r .facts.region)
export BACKUP_REGION=$(curl -H "Authorization: Bearer ${CLOUDSCALE_API_TOKEN}" https://api.cloudscale.ch/v1/regions | jq -r '.[].slug' | grep -v $REGION)
export HIERADATA_REPO_SECRET=$(vault kv get \
-format=json "clusters/kv/lbaas/hieradata_repo_token" | jq -r '.data.data.token')
----

. Compile the catalog for the cluster.
Having the catalog available locally enables us to run Terraform for the cluster to make any required changes.
+
[source,bash]
----
commodore catalog compile "${CLUSTER_ID}"
----

. Configure Terraform secrets
+
[source,bash]
Expand All @@ -69,7 +61,7 @@ for id in 1 2; do
LB_FQDNS[$id]=$(terraform state show "module.cluster.module.lb.cloudscale_server.lb[$(expr $id - 1)]" | grep fqdn | awk '{print $2}' | tr -d ' "\r\n')
done
for lb in ${LB_FQDNS[*]}; do
ssh "${lb}" "sudo grep 'server =' /etc/burp/burp.conf && sudo grep 'ParentZone' /etc/icinga2/constants.conf"
ssh "${lb}" "sudo grep 'server =' /etc/burp/burp.conf && sudo grep 'ParentZone' /etc/icinga2/constants.conf" | tee "../../../$lb.info"
done
----

Expand Down Expand Up @@ -159,6 +151,21 @@ done

. Decommission Puppet-managed LBs according to the https://wiki.vshn.net/display/VT/How+To%3A+Decommission+a+VM[VSHN documentation] (Internal link).
+
[NOTE]
--
The documentation linked above requires some information to be retrieved from the already-deleted load balancers.

If you've been following these instructions, you stored this information into a file earlier:

[source,bash]
----
for lb in ${LB_FQDNS[*]}; do
echo "$lb"
cat "$lb.info"
done
----
--
+
NOTE: Don't forget to remove the LB configuration in the https://git.vshn.net/appuio/appuio_hieradata/-/tree/master/lbaas[APPUiO hieradata] and the https://git.vshn.net/vshn-puppet/nodes_hieradata[nodes hieradata].

. Delete cluster from Lieutenant API (via portal)
Expand All @@ -175,16 +182,6 @@ Go to https://control.vshn.net/syn/lieutenantclusters
+
TIP: This step is required because the csi-cloudscale driver doesn't have time to properly cleanup PVs when the cluster is decommissioned with `terraform destroy`.

. Delete the cluster-backup bucket in the cloudscale.ch project
+
[NOTE]
====
Verify that the cluster backups aren't needed anymore before cleaning up the backup bucket.
Consider extracting the most recent cluster objects and etcd backups before deleting the bucket.
See the xref:how-tos/recover-from-backup.adoc[Recover objects from backup] how-to for instructions.
At this point in the decommissioning process, you'll have to extract the Restic configuration from Vault instead of the cluster itself.
====

. Delete the cluster's API tokens in the cloudscale UI

. Delete Keycloak service (via portal)
Expand Down
Loading