Add a new "hub-experimental" hub image and enable unlisted choices for leap hub

config/clusters/leap/common.values.yaml

@@ -37,6 +37,9 @@ basehub:
             name: LEAP
             url: https://leap-stc.github.io
     hub:
+      image:
+        name: quay.io/2i2c/experimental-hub
+        tag: "0.0.1-0.dev.git.6406.hc1091b1c"
       allowNamedServers: true
       config:
         Authenticator:
@@ -189,6 +192,13 @@ basehub:
                     cpu_limit: 16
             image: &profile_list_profile_options_image
               display_name: Image
+              unlisted_choice: &profile_list_unlisted_choice
+                enabled: True
+                display_name: "Custom image"
+                validation_regex: "^.+:.+$"
+                validation_message: "Must be a valid public docker image, including a tag"
+                kubespawner_override:
+                  image: "{value}"
               choices:
                 pangeo_new:
                   display_name: Base Pangeo Notebook ("2023.05.18")
@@ -268,6 +278,7 @@ basehub:
           profile_options:
             image:
               display_name: Image
+              unlisted_choice: *profile_list_unlisted_choice
               choices:
                 tensorflow_new:
                   display_name: Pangeo Tensorflow ML Notebook ("2023.05.18")

docs/conf.py

@@ -15,6 +15,7 @@
     "sphinx.ext.intersphinx",
     "sphinx_copybutton",
     "sphinx_design",
+    "sphinxcontrib.mermaid",
 ]
 
 intersphinx_mapping = {

docs/howto/custom-jupyterhub-image.md

@@ -1,20 +1,66 @@
 # Modify our custom JupyterHub image
 
-The 2i2c hubs use a custom hub image that is defined in the [helm-charts directory of the infrastructure repository](https://github.com/2i2c-org/infrastructure/tree/HEAD/helm-charts/images/hub).
+The 2i2c hubs use a custom `hub` image that is defined in the [helm-charts directory of the infrastructure repository](https://github.com/2i2c-org/infrastructure/tree/HEAD/helm-charts/images/hub).
+
+## The `hub` image
 
 This custom hub image is built on top of the [jupyterhub/k8s-hub](https://hub.docker.com/r/jupyterhub/k8s-hub) Docker image and configured based on the needs of 2i2c hubs.
 This allows adding and configuring other packages like the [jupyterhub-configurator](https://github.com/yuvipanda/jupyterhub-configurator) or using specific versions of the spawner and authenticator.
 More information about this custom image can be found in the [Dockerfile](https://github.com/2i2c-org/infrastructure/blob/HEAD/helm-charts/images/hub/Dockerfile) itself.
 
-The 2i2c custom hub image is lives at [Quay.io](https://quay.io/repository/2i2c/pilot-hub).
 
-## Updating the hub image
+## The `hub-experimental` image
+
+In addition to the `hub` image, there is also a `hub-experimental` image, that is defined in the [helm-charts directory of the infrastructure repository](https://github.com/2i2c-org/infrastructure/tree/HEAD/helm-charts/images/hub-experimental).
+
+This `hub-experimental` image is very similar to the hub image, but its primary goal is to be used to test changes like new packages, newer package versions, or even unreleased or unmerged versions of them. It can be used to deploy such change just to one or a few specific communities, without having to worry about the instabilities it can generate for other communities.
+
+```{note}
+Both the 2i2c custom `hub` and the `hub-experimental` images live at [Quay.io](https://quay.io/organization/2i2c).
+```
 
-When this hub image needs to be updated, the steps to take are:
+## When to roll out the changes in `hub-experimental` to `hub`
 
-1. Update the [Dockerfile](https://github.com/2i2c-org/infrastructure/blob/HEAD/helm-charts/images/hub/Dockerfile) with any changes wanted
+```{important}
+Our current policy states that a change in `hub-experimental` must be rolled out into `hub`, **within  4 weeks**, otherwise it should be removed. Any issues identified within these 4 weeks, must be fixed it until it's good enough to deploy to all hubs.
+```
 
-2. Commit the changes
+The workflow is the following:
+```{mermaid}
+flowchart TD
+  build_hub_experimental[fa:fa-camera-retro Build a new `hub-experimental` image]-->configure_hub_with_experiment[Configure the hub of at least one community to use the new image]
+  configure_hub_with_experiment-->deploy_hub[fa:fa-rocket Deploy]
+  -- Watch for one week to see how it goes! ---
+  deploy_hub --> condition{{Runs for one week without needing any fixes?}}
+  condition -- Yes --- roll_out[Roll out changes from `hub-experimental` into the `hub` image]
+  condition -- No --- fix[fa:fa-ban Fix issues]
+  fix --> condition
+  roll_out --> deploy_everywhere[fa:fa-rocket Deploy hub image everywhere]
+  -- More testing for hubs that are configured differently ---
+  deploy_everywhere --> end_condition[fa:fa-exclamation `hub-experimental` image is the same as `hub`]
+```
+
+## How to install an unreleased version of a package in the `hub-experimental` image
+
+To install an unreleased package, we will need to install directly from GitHub and not from PyPI:
+
+1. Identify the package and commit you wish to install into the hub image
+
+   ```{important}
+   Specify a full commit hash after `@`, not a branch name. This way, our builds are more reproducible!
+   ```
+
+   ```bash
+   https://github.com/jupyterhub/kubespawner@def501f1d60b8e5629745acb0bcc45b151b1decc
+   ```
+
+2. Update the [requirements.txt](https://github.com/2i2c-org/infrastructure/blob/HEAD/helm-charts/images/hub-experimental/requirements.txt) file to add this package and commit, prefixed by a `git+` on a new row
+
+   ```bash
+   git+https://github.com/jupyterhub/kubespawner@def501f1d60b8e5629745acb0bcc45b151b1decc
+   ```
+
+3. Commit the changes
 
    ```bash
    git add helm-charts/images/hub/Dockerfile
@@ -25,56 +71,71 @@ When this hub image needs to be updated, the steps to take are:
    The commit SHA with be used to generate the image tag.
    ```
 
-3. Rebuild the Docker image and push it to the [Quay.io registry](https://quay.io/repository/2i2c/pilot-hub)
+## How to build and push a new version of the hub image (`hub` or `hub-experimental`)
+
+Rebuild the Docker image and push it to the [Quay.io registry](https://quay.io/repository/2i2c/pilot-hub)
 
-   - Your `@2i2c` address should give you access to push to the Quay.io registry where the hub image lives, but make sure you are logged into quay.io container registry with the right credentials and these creds are configured to have access to <https://quay.io/repository/2i2c/pilot-hub>.
-     Please contact someone at 2i2c for access if this is not the case.
+- Your `@2i2c` address should give you access to push to the Quay.io registry where the hub image lives, but make sure you are logged into quay.io container registry with the right credentials and these creds are configured to have access to <https://quay.io/repository/2i2c/pilot-hub>.
+  Please contact someone at 2i2c for access if this is not the case.
 
-     ```bash
-     docker login quay.io
-     ```
+  ```bash
+  docker login quay.io
+  ```
 
-     ```{seealso}
-     Checkout the [Getting Started with Quay.io](https://docs.quay.io/solution/getting-started.html) docs for more info.
-     ```
+  ```{seealso}
+  Checkout the [Getting Started with Quay.io](https://docs.quay.io/solution/getting-started.html) docs for more info.
+  ```
 
-   - Make sure you have [jupyterhub/chartpress](https://github.com/jupyterhub/chartpress) installed.
+- Make sure you have [jupyterhub/chartpress](https://github.com/jupyterhub/chartpress) installed.
 
-     ```bash
-     pip install chartpress
-     ```
+  ```bash
+  pip install chartpress
+  ```
 
-     This package is also listed under `dev-requirements.txt`, so it should be present if you've installed the dev dependencies.
+  This package is also listed under `dev-requirements.txt`, so it should be present if you've installed the dev dependencies.
 
-   - Make sure you are in the `helm-charts` directory, where the `chartpress.yaml` is located:
+- Make sure you are in the `helm-charts` directory, where the `chartpress.yaml` is located:
 
-     ```bash
-     cd ./helm-charts
-     ```
+  ```bash
+  cd ./helm-charts
+  ```
 
-   - Run chartpress to build the image, push it to the registry and update the basehub helm chart to use the updated image tag
+- Run chartpress to build the image, push it to the registry and update the basehub helm chart to use the updated image tag
 
-     ```bash
-     chartpress --push
-     ```
+  ```bash
+  chartpress --push
+  ```
 
-     ````{note}
-     If you are on macOs with M1, you need to run chartpress with [docker buildx](https://docs.docker.com/build/buildx/) under the hood and specify which platform to use, i.e. `amd64`[^1].
+  ````{note}
+  If you are on macOs with M1, you need to run chartpress with [docker buildx](https://docs.docker.com/build/buildx/) under the hood and specify which platform to use, i.e. `amd64`[^1].
 
-     ```
-      chartpress --push --builder docker-buildx --platform linux/amd64
-     ```
-     ````
+  ```
+  chartpress --push --builder docker-buildx --platform linux/amd64
+  ```
+  ````
 
-   - Commit the changes made by `chartpress` to `helm-charts/basehub/values.yaml`, but discard the changes made to `helm-charts/basehub/Chart.yaml` as the last may cause problems with the `daskhub` dependency mechanism.
+- **If building and pushing the `hub` image**, then commit the changes made by `chartpress` to `helm-charts/basehub/values.yaml`, but discard the changes made to `helm-charts/basehub/Chart.yaml` as the last may cause problems with the `daskhub` dependency mechanism.
 
-     ```bash
-     git add helm-charts/basehub/values.yaml
-     git commit
-     ```
+  ```bash
+  git add helm-charts/basehub/values.yaml
+  git commit
+  ```
 
-```{note}
+- **If building and pushing the `hub-experimental` image**, then discard the changes to both `helm-charts/basehub/values.yaml` and `helm-charts/basehub/Chart.yaml`, because we only want to deploy this image to particular hubs
+
+## How to configure a hub to use the new `hub-experimental` image
+
+You will need to put a config similar to the one below in your hub configuration file:
+
+```
+hub:
+  image:
+    name: quay.io/2i2c/experimental-hub
+    tag: "0.0.1-0.dev.git.6406.hc1091b1c"
+```
+
+```{important}
 The image tag of the of the [jupyterhub/k8s-hub](https://hub.docker.com/r/jupyterhub/k8s-hub) [in the Dockerfile](https://github.com/2i2c-org/infrastructure/blob/HEAD/helm-charts/images/hub/Dockerfile#L9) must match the dependent JupyterHub Helm chart's version as declared in [basehub/Chart.yaml](https://github.com/2i2c-org/infrastructure/blob/master/helm-charts/basehub/Chart.yaml#L14).
 ```
 
-[^1]: <https://cloudolife.com/2022/03/05/Infrastructure-as-Code-IaC/Container/Docker/Docker-buildx-support-multiple-architectures-images/>
+[^1]: <https://cloudolife.com/2022/03/05/Infrastructure-as-Code-IaC/Container/Docker/Docker-buildx-support-multiple-architectures-images/>

docs/howto/features/allow-unlisted-profile-choice.md

@@ -0,0 +1,34 @@
+# Allow users to setup custom, free-form user profile choices
+
+Sometimes it is useful to allow users to specify their own, free-form choice for an option.
+We can make this possible by enabling and configuring [`KubeSpawner.profile_list.<profile>.profile_options.<option>.unlisted_choice`](https://jupyterhub-kubespawner.readthedocs.io/en/latest/spawner.html#kubespawner.KubeSpawner.profile_list).
+
+If enabled, when the user selects “Other” as a choice, a free form input is enabled. An optional regex can be configured that the free form input should match - eg. `^pangeo/.*$`.
+
+Example configuration that allows a custom free-form, user specified image:
+
+```yaml
+jupyterhub:
+   singleuser:
+      profileList:
+        - display_name: "CPU only"
+          description: "Start a container limited to a chosen share of capacity on a node of this type"
+          profile_options:
+            image:
+              display_name: Image
+              unlisted_choice:
+                enabled: True
+                display_name: "Custom image"
+                validation_regex: "^.+:.+$"
+                validation_message: "Must be an image location, matching ^.+:.+$"
+                kubespawner_override:
+                  image: "{value}"
+              choices:
+                pangeo_new:
+                  display_name: Base Pangeo Notebook ("2023.05.18")
+                  default: true
+                  slug: "pangeo_new"
+                  kubespawner_override:
+                    image: "pangeo/pangeo-notebook:2023.05.18"
+                pangeo_old: (...)
+```

docs/howto/features/index.md

@@ -47,5 +47,5 @@ ephemeral.md
 :maxdepth: 2
 :caption: Other
 dedicated-nodepool.md
-custom-jupyterhub-image.md
+allow-unlisted-profile-choice.md
 ```

docs/index.md

@@ -69,6 +69,7 @@ deployed occasionally as a specific addition.
 :caption: How-to guides
 howto/features/index.md
 howto/bill.md
+howto/custom-jupyterhub-image.md
 howto/manage-domains/index.md
 howto/grafana-github-auth.md
 howto/update-env.md

docs/requirements.txt

@@ -7,3 +7,4 @@ sphinx-copybutton
 sphinx-design
 # Pin pydate-sphinx-theme
 git+https://github.com/2i2c-org/sphinx-2i2c-theme
+sphinxcontrib-mermaid==0.9.2

docs/topic/features.md

@@ -1,4 +1,4 @@
-(hub-features)=
+(hub-features-summary)=
 # Features available on the hubs
 
 This document is a concise description of various features we can

helm-charts/chartpress.yaml

@@ -4,3 +4,7 @@ charts:
     images:
       hub:
         valuesPath: jupyterhub.hub.image
+  - name: basehub
+    images:
+      hub-experimental:
+        imageName: quay.io/2i2c/experimental-hub

helm-charts/images/hub-experimental/Dockerfile

@@ -0,0 +1,25 @@
+# This image tag should match the dependent JupyterHub Helm chart's version as
+# declared in basehub/Chart.yaml.
+#
+# If you make an update to this tag and the JupyterHub Helm chart's version,
+# then first commit those changes.
+# Then, from the `helm-charts` directory where the `chartpress.yaml` is located,
+# perform `chartpress --push` with your quay.io container registry credentials
+# configured to have access to https://quay.io/repository/2i2c/pilot-hub.
+#
+# Note: if on macOs with M1, you need to run chartpress with docker buildx under the hood and specify
+# the architecture to use.
+# `chartpress --push --builder docker-buildx --platform linux/amd64`
+# Ref: https://cloudolife.com/2022/03/05/Infrastructure-as-Code-IaC/Container/Docker/Docker-buildx-support-multiple-architectures-images/
+#
+FROM jupyterhub/k8s-hub:3.0.0-beta.1
+
+COPY requirements.txt /tmp/
+
+RUN pip install -r /tmp/requirements.txt
+
+USER root
+RUN mkdir -p /usr/local/etc/jupyterhub-configurator
+
+COPY jupyterhub_configurator_config.py /usr/local/etc/jupyterhub-configurator/jupyterhub_configurator_config.py
+USER $NB_USER

helm-charts/images/hub-experimental/jupyterhub_configurator_config.py

@@ -0,0 +1,15 @@
+import json
+import os
+from glob import glob
+
+CONFIGURATOR_BASE_PATH = "/usr/local/etc/jupyterhub-configurator"
+
+schema_files = sorted(glob(os.path.join(CONFIGURATOR_BASE_PATH, "*.schema.json")))
+
+schemas = {}
+
+for sf in schema_files:
+    with open(sf) as f:
+        schemas[sf] = json.load(f)
+
+c.Configurator.schemas = schemas

helm-charts/images/hub-experimental/requirements.txt

@@ -0,0 +1,3 @@
+git+https://github.com/yuvipanda/jupyterhub-configurator@ed7e3a0df1e3d625d10903ef7d7fd9c2fbb548db
+# Brings on using `unlisted_choice` in profile options per https://github.com/2i2c-org/infrastructure/issues/2146
+git+https://github.com/jupyterhub/kubespawner@def501f1d60b8e5629745acb0bcc45b151b1decc