R image failing to correctly activate jupyter server extension

[grallewellyn]

Seems to potentially be happening as a result in the upgrade of maap-py from v3.1.4 and more specifically this commit: MAAP-Project/maap-py@f9a89e0

We also think it might be that jupyter server extension isn't installing all of its requirements: MAAP-Project/jupyter-server-extension#4 namely xmltodict

Right now I am testing adding in all these requirements to the setup.py of jupyter server extension, but if I add in the packages as

jupyter_packaging==0.12.3
jupyter_server==2.13.0
nbformat==5.10.3
notebook==7.1.2
pytest==8.1.1
PyYAML==6.0.1
Requests==2.31.0
setuptools==68.2.2
tornado==6.4
xmltodict==0.13.0

The build fails with

Type Error: The "path" argument must be of type string or an instance of Buffer or URL. Received undefined
#30 124.3     at Object.mkdir (node:fs:1359:10)
#30 124.3     at /opt/conda/envs/r/lib/python3.10/site-packages/jupyterlab/staging/yarn.js:314:6147
#30 124.3     at new Promise (<anonymous>)
#30 124.3     at $t.mkdirPromise (/opt/conda/envs/r/lib/python3.10/site-packages/jupyterlab/staging/yarn.js:314:6115)
#30 124.3     at Rt.setup (/opt/conda/envs/r/lib/python3.10/site-packages/jupyterlab/staging/yarn.js:426:1556)
#30 124.3     at Rt.find (/opt/conda/envs/r/lib/python3.10/site-packages/jupyterlab/staging/yarn.js:426:720)
#30 124.3     at ou.execute (/opt/conda/envs/r/lib/python3.10/site-packages/jupyterlab/staging/yarn.js:453:512)
#30 124.3     at async ou.validateAndExecute (/opt/conda/envs/r/lib/python3.10/site-packages/jupyterlab/staging/yarn.js:345:664)
#30 124.3     at async Un.run (/opt/conda/envs/r/lib/python3.10/site-packages/jupyterlab/staging/yarn.js:359:2057)
#30 124.3     at async Un.runExit (/opt/conda/envs/r/lib/python3.10/site-packages/jupyterlab/staging/yarn.js:359:2241)
#30 124.3 
#30 ERROR: process "conda run -n r /bin/bash -c jupyter lab build &&     jupyter lab clean &&     jlpm cache clean &&     npm cache clean --force &&     rm -rf $HOME/.node-gyp &&     rm -rf $HOME/.local" did not complete successfully: exit code: 1

Unpinning the versions is a successful build, but fails with Error: Failed to run the workspace: "Server 'jupyter' in container 'ws/s3fs' not available." when launching in the ADE which means jupyter_server needs to be pinned to 2.12.5 (I am trying that right now)

Potentially try adding: git+https://github.com/MAAP-Project/[email protected]#egg=maapPy to install_requires

The easiest way to see this error manifesting is that you cannot view your jobs in DIT

[chuckwondo]

@grallewellyn, I suspect you might get better results with a slight tweak to this command:

conda run -n r /bin/bash -c jupyter lab build && \
    jupyter lab clean && \
    jlpm cache clean && \
    npm cache clean --force && \
    rm -rf $HOME/.node-gyp && \
    rm -rf $HOME/.local

Throwing /bin/bash into the mix with conda run can create odd/unintended behavior, so I suggest the following instead, getting rid of /bin/bash and inserting another conda run in front of jupyter lab clean:

conda run -n r jupyter lab build && \
    conda run -n r jupyter lab clean && \
    jlpm cache clean && \
    npm cache clean --force && \
    rm -rf $HOME/.node-gyp && \
    rm -rf $HOME/.local

[grallewellyn]

Our code is already

RUN jupyter lab build && \
    jupyter lab clean && \
    jlpm cache clean && \
    npm cache clean --force && \
    rm -rf $HOME/.node-gyp && \
    rm -rf $HOME/.local

without the bin/bash

From this build

Maybe there is another command we have to add to avoid it adding the bin/bash?

[chuckwondo]

Seems to be coming from here: https://github.com/MAAP-Project/maap-workspaces/blob/develop/jupyterlab3/docker/Dockerfile#L10-L20

[grallewellyn]

Okay, so change to

FROM jupyterlab_base as workspace_base_pangeo
SHELL ["conda", "run", "-n", "pangeo", "-c"]

? @anilnatha What are your thoughts on this since you added those lines?

[chuckwondo]

You would also remove "-c", which is an option for "/bin/bash".

[chuckwondo]

However, I'm not confident that will fix the issue, because the last line of the error shows the following (as I originally mentioned above):

#30 ERROR: process "conda run -n r /bin/bash -c jupyter lab build &&     jupyter lab clean &&     jlpm cache clean &&     npm cache clean --force &&     rm -rf $HOME/.node-gyp &&     rm -rf $HOME/.local" did not complete successfully: exit code: 1

Notice that conda run shows up only at the beginning. However, that will only apply to jupyter lab build (the very first command), not any subsequent commands, so if it needs to be applied to other commands, that doesn't do the trick.

Instead, I suggest not using conda run within the SHELL lines. Instead, put conda run on the RUN line itself, and use SHELL only for bash.

For example:

SHELL ["/bin/bash", "-c"]

RUN conda run -n r jupyter lab build && \
    conda run -n r jupyter lab clean && \
    jlpm cache clean && \
    npm cache clean --force && \
    rm -rf $HOME/.node-gyp && \
    rm -rf $HOME/.local

Notice that conda run precedes the 2 distinct jupyter lab commands.

[grallewellyn]

How would the conda environment activate then if you are suggesting to change the lines to

FROM jupyterlab_base as workspace_base_vanilla
SHELL ["/bin/bash", "-c"]

Or maybe you meant

FROM jupyterlab_base as workspace_base_vanilla
... rest of Dockerfile ...
SHELL ["/bin/bash", "-c"]

RUN conda run -n r jupyter lab build && \
    conda run -n r jupyter lab clean && \
    jlpm cache clean && \
    npm cache clean --force && \
    rm -rf $HOME/.node-gyp && \
    rm -rf $HOME/.local

[chuckwondo]

I didn't realize that was your intent. You could simply drop the "/bin/bash" and "-c" from the SHELL, as mentioned earlier, and see how that works. Perhaps that will work okay, but it seems a bit dicey to use conda run as the "shell".

[grallewellyn]

Changing to

FROM jupyterlab_base as workspace_base_vanilla
SHELL ["conda", "run", "-n", "vanilla"]

Fails: https://repo.dit.maap-project.org/root/maap-workspaces/-/jobs/11751

If you think removing "/bin/bash" from the SHELL is a change we should implement, I will make a new ticket because Brian Satorius found a fix to get the R workspace working again without making this change

[chuckwondo]

I'd like to see what @bsatoriu came up with.

Generally speaking though, I don't think we should be taking this approach. The point of using conda run is not for it to be used as a means of activating a "default" conda env in the container. Users should be using conda run themselves, in place of conda activate or source activate.

[grallewellyn]

Brian's work is in this branch: https://github.com/MAAP-Project/maap-workspaces/tree/bsatoriu-patch-dit

I created a ticket for this here: #967