Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Configure dask workers to use the new nodegroups #5161

Merged
merged 8 commits into from
Nov 20, 2024
Merged

Configure dask workers to use the new nodegroups #5161

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
8 commits
Select commit Hold shift + click to select a range
00df47e
gridsst: Configure dask workers to use the new nodegroups
GeorgianaElena Nov 20, 2024
9ff114b
opensci: Configure dask workers to use the new nodegroups
GeorgianaElena Nov 20, 2024
1833413
smithsonian: Configure dask workers to use the new nodegroups
GeorgianaElena Nov 20, 2024
1bd61c9
earthscope: Configure dask workers to use the new nodegroups
GeorgianaElena Nov 20, 2024
aeb83ae
Fix typo
GeorgianaElena Nov 20, 2024
de6d52a
Only setup dask-gateway for daskhubs
GeorgianaElena Nov 20, 2024
56357bd
Indent correctly the setup and add missing key
GeorgianaElena Nov 20, 2024
30cbafb
Fix typo
GeorgianaElena Nov 20, 2024
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
21 changes: 11 additions & 10 deletions config/clusters/2i2c-aws-us/dask-staging.values.yaml
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -42,13 +42,14 @@ basehub:
GitHubOAuthenticator:
oauth_callback_url: "https://dask-staging.aws.2i2c.cloud/hub/oauth_callback"

dask-gateway:
gateway:
scheduler:
extraPodConfig:
nodeSelector:
2i2c/hub-name: dask-staging
worker:
extraPodConfig:
nodeSelector:
2i2c/hub-name: dask-staging
dask-gateway:
gateway:
backend:
scheduler:
extraPodConfig:
nodeSelector:
2i2c/hub-name: dask-staging
worker:
extraPodConfig:
nodeSelector:
2i2c/hub-name: dask-staging
21 changes: 11 additions & 10 deletions config/clusters/2i2c-aws-us/ncar-cisl.values.yaml
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -253,13 +253,14 @@ basehub:
extra_resource_limits:
nvidia.com/gpu: "1"

dask-gateway:
gateway:
scheduler:
extraPodConfig:
nodeSelector:
2i2c/hub-name: ncar-cisl
worker:
extraPodConfig:
nodeSelector:
2i2c/hub-name: ncar-cisl
dask-gateway:
gateway:
backend:
scheduler:
extraPodConfig:
nodeSelector:
2i2c/hub-name: ncar-cisl
worker:
extraPodConfig:
nodeSelector:
2i2c/hub-name: ncar-cisl
21 changes: 11 additions & 10 deletions config/clusters/2i2c-aws-us/showcase.values.yaml
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -249,13 +249,14 @@ basehub:
extra_resource_limits:
nvidia.com/gpu: "1"

dask-gateway:
gateway:
scheduler:
extraPodConfig:
nodeSelector:
2i2c/hub-name: showcase
worker:
extraPodConfig:
nodeSelector:
2i2c/hub-name: showcase
dask-gateway:
gateway:
backend:
scheduler:
extraPodConfig:
nodeSelector:
2i2c/hub-name: showcase
worker:
extraPodConfig:
nodeSelector:
2i2c/hub-name: showcase
11 changes: 11 additions & 0 deletions config/clusters/earthscope/prod.values.yaml
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -28,3 +28,14 @@ basehub:
2i2c/hub-name: prod
extraEnv:
SCRATCH_BUCKET: s3://earthscope-scratch/$(JUPYTERHUB_USER)
dask-gateway:
gateway:
backend:
scheduler:
extraPodConfig:
nodeSelector:
2i2c/hub-name: prod
worker:
extraPodConfig:
nodeSelector:
2i2c/hub-name: prod
11 changes: 11 additions & 0 deletions config/clusters/earthscope/staging.values.yaml
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -30,3 +30,14 @@ basehub:
2i2c/hub-name: staging
extraEnv:
SCRATCH_BUCKET: s3://earthscope-scratch-staging/$(JUPYTERHUB_USER)
dask-gateway:
gateway:
backend:
scheduler:
extraPodConfig:
nodeSelector:
2i2c/hub-name: staging
worker:
extraPodConfig:
nodeSelector:
2i2c/hub-name: staging
11 changes: 11 additions & 0 deletions config/clusters/gridsst/prod.values.yaml
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -15,3 +15,14 @@ basehub:
singleuser:
nodeSelector:
2i2c/hub-name: prod
dask-gateway:
gateway:
backend:
scheduler:
extraPodConfig:
nodeSelector:
2i2c/hub-name: prod
worker:
extraPodConfig:
nodeSelector:
2i2c/hub-name: prod
11 changes: 11 additions & 0 deletions config/clusters/gridsst/staging.values.yaml
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -15,3 +15,14 @@ basehub:
singleuser:
nodeSelector:
2i2c/hub-name: staging
dask-gateway:
gateway:
backend:
scheduler:
extraPodConfig:
nodeSelector:
2i2c/hub-name: staging
worker:
extraPodConfig:
nodeSelector:
2i2c/hub-name: staging
21 changes: 11 additions & 10 deletions config/clusters/jupyter-meets-the-earth/prod.values.yaml
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -32,13 +32,14 @@ basehub:
nodeSelector:
2i2c/hub-name: prod
node.kubernetes.io/instance-type: m5.xlarge
dask-gateway:
gateway:
scheduler:
extraPodConfig:
nodeSelector:
2i2c/hub-name: prod
worker:
extraPodConfig:
nodeSelector:
2i2c/hub-name: prod
dask-gateway:
gateway:
backend:
scheduler:
extraPodConfig:
nodeSelector:
2i2c/hub-name: prod
worker:
extraPodConfig:
nodeSelector:
2i2c/hub-name: prod
21 changes: 11 additions & 10 deletions config/clusters/jupyter-meets-the-earth/staging.values.yaml
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -17,13 +17,14 @@ basehub:
2i2c/hub-name: staging
extraEnv:
SCRATCH_BUCKET: s3://jupyter-meets-the-earth-scratch-staging/$(JUPYTERHUB_USER)
dask-gateway:
gateway:
scheduler:
extraPodConfig:
nodeSelector:
2i2c/hub-name: staging
worker:
extraPodConfig:
nodeSelector:
2i2c/hub-name: staging
dask-gateway:
gateway:
backend:
scheduler:
extraPodConfig:
nodeSelector:
2i2c/hub-name: staging
worker:
extraPodConfig:
nodeSelector:
2i2c/hub-name: staging
8 changes: 8 additions & 0 deletions config/clusters/opensci/big-binder.values.yaml
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -3,6 +3,14 @@ dask-gateway:
gateway:
backend:
imagePullSecrets: [{ name: image-pull-secret }]
scheduler:
extraPodConfig:
nodeSelector:
2i2c/hub-name: big-binder
worker:
extraPodConfig:
nodeSelector:
2i2c/hub-name: big-binder
jupyterhub:
ingress:
hosts: [hub.big.binder.opensci.2i2c.cloud]
Expand Down
11 changes: 11 additions & 0 deletions config/clusters/opensci/climaterisk.values.yaml
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -39,3 +39,14 @@ jupyterhub:
cpu:
guarantee: 1.8625
limit: 3.725
dask-gateway:
gateway:
backend:
scheduler:
extraPodConfig:
nodeSelector:
2i2c/hub-name: climaterisk
worker:
extraPodConfig:
nodeSelector:
2i2c/hub-name: climaterisk
11 changes: 11 additions & 0 deletions config/clusters/smithsonian/prod.values.yaml
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -12,3 +12,14 @@ basehub:
singleuser:
nodeSelector:
2i2c/hub-name: prod
dask-gateway:
gateway:
backend:
scheduler:
extraPodConfig:
nodeSelector:
2i2c/hub-name: prod
worker:
extraPodConfig:
nodeSelector:
2i2c/hub-name: prod
11 changes: 11 additions & 0 deletions config/clusters/smithsonian/staging.values.yaml
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -12,3 +12,14 @@ basehub:
singleuser:
nodeSelector:
2i2c/hub-name: staging
dask-gateway:
gateway:
backend:
scheduler:
extraPodConfig:
nodeSelector:
2i2c/hub-name: staging
worker:
extraPodConfig:
nodeSelector:
2i2c/hub-name: staging