Update commcare cloud setup instructions #6458

gherceg · 2025-01-03T19:23:59Z

I've run into various issues while setting up monoliths and am attempting to clarify those instructions in this PR.

This also notably updates documentation to be aligned with our removal of the suggestion to copy a private key to the server, and instead use ssh -A when/where appropriate.

Review by commit.

Environments Affected

None

This is necessary so that COMMCARE_CLOUD_ENVIRONMENTS environment variable is set properly before running init-ansible.

bootstrap-users sets up the expected ssh config based on public keys previously added in these instructions. Once run, the user should then exit and re-ssh into the machine using the -A option to ensure agent forwarding is enabled.

This is removed once bootstrap-users is run

mkangia · 2025-01-05T12:12:25Z

docs/source/installation/2-manual-install.rst

-
-        $ sudo update-alternatives --install /usr/bin/python python /usr/bin/python3 10
-
-


is this not needed anymore?

I used this one recently when trying to do a test install.

Oh I see, its moved.

Given that we specify ubuntu 22.04 at the moment, 3.10 will be the system version, so it seems fine to simplify the step to just run this rather than specify the condition.

mkangia · 2025-01-06T09:21:52Z

docs/source/installation/2-manual-install.rst

@@ -111,6 +111,12 @@ reference.

        PasswordAuthentication yes

+    To allow keyboard interactive authentication, ensure


I guess this is needed in specific cases and not all interactive authentication. I definitely could do an install without this change.

Found this useful reference: https://serverfault.com/questions/189643/sshd-blocking-password-authentication

Yeah agreed it isn't necessary all of the time. Just checking that you agree that it isn't worth specifying that in the instructions given that it will be reverted in a later setup step anyway?

I think if someone gets stuck at this step like you did, they are doing something special.

Seems fine to leave it if you think it could be useful for someone to make the initial root password step work, with a comment that "If needed you can enable keyboard interactive authentication with", though there are so many steps here that one would just go about copying pasting commands without really reading the instructions well, which is why it would be nice to keep commands limited and exclude anything special.
So, This could go either also go here

Agreed that this later gets cleaned up.

mkangia · 2025-01-06T09:22:06Z

docs/source/installation/2-manual-install.rst


 ::

-   $ eval `ssh-agent`
-   $ ssh-add ~/.ssh/id_rsa
+    $ commcare-cloud cluster bootstrap-users


AmitPhulera

Great improvements, clearly makes things lot clearer. Thanks for the PR. Just have a few questions/suggestions otherwise this looks gtg.

docs/source/installation/2-manual-install.rst

AmitPhulera · 2025-01-06T10:05:52Z

docs/source/installation/2-manual-install.rst

@@ -467,13 +464,13 @@ Install CommCare Cloud

       $ cp ~/commcare-cloud/src/commcare_cloud/config.example.py ~/commcare-cloud/src/commcare_cloud/config.py

-    Update the known hosts file
+    Update the known hosts file (substituting your environment name if necessary)

    ::

       $ commcare-cloud cluster update-local-known-hosts


I know we have used cluster as environment name in the entire doc but would it be clearer if we use which would indicate that this has to be replaced?

Suggested change

$ commcare-cloud cluster update-local-known-hosts

$ commcare-cloud <env> update-local-known-hosts

btw I am totally fine with the way it is right now it was just a thought that I wanted to share with you.

Yeah I do want to make this change. I also want to make a sweeping change to the $ syntax before each command as it makes copy/pasting annoying, so was planning to do this in a separate PR if that is ok with you.

Yeah. Totally fine by me.

AmitPhulera · 2025-01-06T10:09:58Z

docs/source/installation/2-manual-install.rst

+        $ nano ~/.profile
+
+    Paste the following before the line that sources the init-ansible script.
+
+    ::
+
+        source ~/.commcare-cloud/load_config.sh


Any specific case where this approach better than the previous one?

Based on the order of instructions, load_config.sh was being run after init-ansible, but load_config plays a critical role in exporting the COMMCARE_CLOUD_ENVIRONMENTS variable to point to the right environments directory. I know this change complicates the setup a bit, but hopefully results in fewer issues down the road. Does that answer your question?

The "before ... init-ansible" instruction appears to contradict the output of manage-commcare-cloud configure, which says to append to ~/.profile.

commcare-cloud/src/commcare_cloud/manage_commcare_cloud/configure.py

Lines 97 to 98 in d6adf3b

puts(color_notice("Append the following to your ~/.profile:"))

puts(color_code("source ~/.commcare-cloud/load_config.sh"))

load_config.sh was being run after init-ansible, but load_config plays a critical role in exporting the COMMCARE_CLOUD_ENVIRONMENTS variable to point to the right environments directory

Does that order matter? That is, does init-ansible need COMMCARE_CLOUD_ENVIRONMENTS to be set before it is run?

Another option would be to have the init-ansible script source ~/.commcare-cloud/load_config.sh if it exists, and skip the instruction to add it to .profile.

Yeah upon closer inspection, I may have incorrectly assumed this was the cause of an issue I was facing. It seems fair to say that the init-ansible script does not use the COMMCARE_CLOUD_ENVIRONMENTS variable for anything other than how it is used in update_code.sh. I think I know where I went wrong, but I'll try to run through the setup steps without this change just to make sure.

Ok it seems fine. Reverted in 17240b4.

gherceg · 2025-01-06T15:56:28Z

Possibly a more extreme change, but I'm tempted to remove support for running bootstrap-users as part of the deploy-stack command entirely, since bootstrap users is a one-off command that will change the state of user permissions on the system, and inevitably result in needed to re-run deploy-stack from a different user config, at least in my experience.

millerdev · 2025-01-06T19:41:18Z

docs/source/installation/2-manual-install.rst

+        $ nano ~/.profile
+
+    Paste the following before the line that sources the init-ansible script.
+
+    ::
+
+        source ~/.commcare-cloud/load_config.sh


The "before ... init-ansible" instruction appears to contradict the output of manage-commcare-cloud configure, which says to append to ~/.profile.

commcare-cloud/src/commcare_cloud/manage_commcare_cloud/configure.py

Lines 97 to 98 in d6adf3b

puts(color_notice("Append the following to your ~/.profile:"))

puts(color_code("source ~/.commcare-cloud/load_config.sh"))

load_config.sh was being run after init-ansible, but load_config plays a critical role in exporting the COMMCARE_CLOUD_ENVIRONMENTS variable to point to the right environments directory

Does that order matter? That is, does init-ansible need COMMCARE_CLOUD_ENVIRONMENTS to be set before it is run?

This reverts commit 3d9717c.

Co-authored-by: Amit Phulera <[email protected]>

gherceg added 6 commits January 3, 2025 14:11

General minor changes to wording and ordering

4e48c2f

Don't suggest ansible password for vault file

26d8044

Ensure load_config.sh is run before init-ansible

3d9717c

This is necessary so that COMMCARE_CLOUD_ENVIRONMENTS environment variable is set properly before running init-ansible.

Remove outdated instruction

3a840ee

Modify initial steps to deploy HQ

01c3af2

bootstrap-users sets up the expected ssh config based on public keys previously added in these instructions. Once run, the user should then exit and re-ssh into the machine using the -A option to ensure agent forwarding is enabled.

Ensure initial ssh config accepts interactive auth

9ce97bc

This is removed once bootstrap-users is run

gherceg changed the title ~~Gh/docs/install~~ Update commcare cloud setup instructions Jan 3, 2025

gherceg requested review from millerdev, kaapstorm, AmitPhulera and mkangia January 3, 2025 19:24

Use su instead

47c96bb

mkangia reviewed Jan 5, 2025

View reviewed changes

mkangia reviewed Jan 6, 2025

View reviewed changes

AmitPhulera reviewed Jan 6, 2025

View reviewed changes

millerdev approved these changes Jan 6, 2025

View reviewed changes

gherceg and others added 2 commits January 6, 2025 16:15

Revert "Ensure load_config.sh is run before init-ansible"

17240b4

This reverts commit 3d9717c.

Fix "Coud" typo

e347f5d

Co-authored-by: Amit Phulera <[email protected]>

millerdev approved these changes Jan 6, 2025

View reviewed changes

AmitPhulera approved these changes Jan 7, 2025

View reviewed changes

gherceg merged commit b03d359 into master Jan 20, 2025
2 checks passed

gherceg deleted the gh/docs/install branch January 20, 2025 17:02

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Update commcare cloud setup instructions #6458

Update commcare cloud setup instructions #6458

gherceg commented Jan 3, 2025

mkangia Jan 5, 2025

mkangia Jan 6, 2025

gherceg Jan 6, 2025

mkangia Jan 6, 2025

gherceg Jan 6, 2025

mkangia Jan 8, 2025

mkangia Jan 6, 2025

AmitPhulera left a comment

AmitPhulera Jan 6, 2025

gherceg Jan 6, 2025

AmitPhulera Jan 7, 2025

AmitPhulera Jan 6, 2025

gherceg Jan 6, 2025

millerdev Jan 6, 2025

millerdev Jan 6, 2025

gherceg Jan 6, 2025

gherceg Jan 6, 2025

gherceg commented Jan 6, 2025

millerdev Jan 6, 2025


		$ sudo update-alternatives --install /usr/bin/python python /usr/bin/python3 10

		@@ -111,6 +111,12 @@ reference.

		PasswordAuthentication yes

		To allow keyboard interactive authentication, ensure

	$ commcare-cloud cluster update-local-known-hosts
	$ commcare-cloud <env> update-local-known-hosts

	puts(color_notice("Append the following to your ~/.profile:"))
	puts(color_code("source ~/.commcare-cloud/load_config.sh"))

Update commcare cloud setup instructions #6458

Update commcare cloud setup instructions #6458

Conversation

gherceg commented Jan 3, 2025

Environments Affected

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

AmitPhulera left a comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

gherceg commented Jan 6, 2025

Choose a reason for hiding this comment