Allow installing system packages into jupyter-labs

[mguidon]

Long story short

jovyan is not in the sudoer group. Nobody can use the package manager. Once we are completely isolated (dynamic sidecar) we might allow that. There are several options:

1. Enable it always
2. Enable it via GRANT_SUDO=yes
3. Have a hook in the containers entrypoint that reads the packages to be installed from a file.

The latter would make it persistent. This came in via discussion with @colinRawlings

[pcrespov]

Ello. To consider in your new cookie cutter? Otherwise pls close

[elisabettai]

@pcrespov, I'd like to have a look to this, yes, but I won't go into the direction of giving the standard jupyterlab user (jovyan) root access.

I think the way to go would be to set pip (or conda, or what we need for R and Julia) so that the jovyan user installs things in a specific directory under workspace. So when the oSPARC users come back to their study, they will find their environments and packages without the need to reinstall.

The main caveat I have in mind is that the workspace will easily grow very big and then the study might take long to open, up to the point when the current behavior (i.e. installing packages all the time) is more efficient than "saving" the packages in workspace. I might miss some details, or maybe @GitHK has some other caveats in mind.

[GitHK]

Regarding root access. I would not go into that direction at all. If people are concern of having web applications running as root access. Immagine how bad it is if the user of the jupuyter-lab has a terminal with root access.

Regarding installing packages. I'm not sure how we might end up guaranteeing that the correct data is backed up. I think the current situation (people always install dependencies) is still the way to go.
If they want something special they ask or create a new service with the dependencies installed.

[pcrespov]

@elisabettai AFAIK now we control the size of the workspace folder. In that sense, i do not see a difference between having extra packages installed vs having more data files in the workspace folder. So for the moment I would go that way.

[mguidon]

I think this issue was more meant for system packages (e.g. apt, but that might be dangerous) For python packages, can we do some measurements?

1. pip install from scratch into non-workspace
2. pip install into workspace
3. pip install using cache that is in workspace (i guess those a tarballs)
4. same thing with uv

[elisabettai]

Also, for a bit more context, in the cookiecutter for Jupyters that I am making, users can setup their environments in python, R or Julia, by providing a requirement file, so they can build a service with whatever dependencies they want (and customize the Dockerfile if needed).

I am not sure I fully understand the 3 points you mention @mguidon, but we can discuss offline.

And I was not aware we control the size of workspace, how big is it e.g. for jupyter-math on osparc.io, @GitHK?

[mguidon]

We control the workspace only for s4l-lite