AWS cost attribution: split dask workers of each hub in all clusters into their own nodegroups

[GeorgianaElena]

For each cluster in the list below, perform the following actions:

Tasks

Beta Give feedback

1. Create the same set of dask worker nodes for each hub in a cluster
2. Configure each of the hubs so their dask workers only spawn on the nodepools designated for them
3. Tag each of the nodepool with 2i2c:hub-name to match the name of the hub
4. Tag each of the dask nodepools with 2i2c:node-purpose set to worker

• 2i2c-aws-us 2i2c-aws-us: Nodegroup split and k8s update #5106
• catalystproject-africa
• gridsst
• jupyter-health
• kitware
• nasa-cryo
• opensci
• smithsonian
• strudel
• ubc-eoas
• victor
• nmfs-openscapes
• openscapes
• earthscope
• jupyter-meets-the-earth
• nasa-ghg
• nasa-veda
• projectpythia (they are using other cost allocation system https://github.com/2i2c-org/meta/issues/1612)

Reference

See #4485

• Users across these hubs will spawn dask workers on to their own nodepools, and not share them with users on other hubs.
• There is a tag 2i2c:hub-name, 2i2c:node-purpose on nodes spawned by dask-gateway. You can verify this by looking at EC2 instances on the AWS console.
• the cost of each hub shows up in its respective grafana