AWS cost attribution: documentation about opting in to have hub specific infra for compute and home storage

[consideRatio]

Historical context

In #4928 (comment), it was decided to not split compute or home storage per hub as part of aws cost allocation cluster rollout tracked by #4878.

Currently this split has only happened for:

• openscapes
• partly for victor: nodepools were split, but not home storage

Technical context

1. Splitting the nodegroups and home storage of existing clusters is not a simple operation and requires care.

We don't have documentation about this, but the following issues can serve as guidelines on what needs to have for all clusters:

• Move each hub to its own nodegroup on the openscapes cluster #4482
• Move dask-gateway workers to their own nodegroups on the openscapes cluster #4485
• Switch staging hub on openscapes hub to its own EFS #4461
• Make new EFS and switch workshop hub to it on openscapes #4475
• Add tags to EFS filesystems on openscapes cluster #4464

2. Splitting nodegroups per hub will be less cost efficient and will increase the average startup time

Hubs will not share nodes anymore, which means node packing will be less efficient and there might be more users on average waiting for a new node to come up than before.

Proposal

Splitting the compute and home storage cost per hub is presented as an opt-in feature to communities and we only do it for
new and existing communities that explicitly request it.

Definition of Done

Tasks

Preview Give feedback

1. get feedback on the proposal above from @yuvipanda
2. create community facing documentation about the pros and cons of having compute and home storage cost split per hub
3. create engineering facing documentation about how to split compute and home storage cost per hub

[GeorgianaElena]

I've refined this and added it to the refined column on the P&S board

[GeorgianaElena]

Closing this in favor of #5074 #5075, #5077, #5078