Production (#3)

* docs: improve README (eduNEXT#56)

* improve install instructions
* include custom image section
* enhance description

---------

Co-authored-by: Maria Grimaldi <[email protected]>

* feat: redwood support (eduNEXT#54)

Add two new variables to enable the use of custom versions/forks of
`edunext/codejailservice`.

BREAKING CHANGE: the default Python version of the sandbox environment
has been bumped to 3.11. This change alongside the upgrade of the SciPy
and NumPy dependencies may cause some instructor code to fail.

---------

Co-authored-by: Diana Olarte <[email protected]>
Co-authored-by: Maria Grimaldi <[email protected]>
Co-authored-by: Moisés González <[email protected]>

.gitignore

@@ -7,7 +7,7 @@ __pycache__/
 TODO
 # Distribution / packaging
 .Python
-build/
+/build/
 develop-eggs/
 dist/
 downloads/

CHANGELOG.rst

@@ -14,6 +14,21 @@ Change Log
 Unreleased
 ~~~~~~~~~~
 
+[18.0.0] - 2024-06-26
+~~~~~~~~~~~~~~~~~~~~~
+* feat: redwood support
+
+  Add two new variables to enable the use of custom versions/forks of
+  `edunext/codejailservice`.
+
+  BREAKING CHANGE: the default Python version of the sandbox environment
+  has been bumped to 3.11. This change alongside the upgrade of the SciPy
+  and NumPy dependencies may cause some instructor code to fail.
+
+[17.0.2] - 2024-05-24
+~~~~~~~~~~~~~~~~~~~~~
+* fix: use tutor config on codejail service and add patches to edit it DS-894 (#55)
+
 [17.0.1] - 2023-11-20
 ~~~~~~~~~~~~~~~~~~~~~
 

README.rst

@@ -13,27 +13,37 @@ Depending on the runtime of your code you might need to extend the timeout time
 Codejail plugin for `Tutor`_
 ============================
 
-Tutor plugin that enables execution of untrusted code in secure sandboxes using an external `service`_ based on the `codejail`_ library.
+Tutor plugin that configures and runs a `Codejail Service`_ using a REST API. `Codejail`_ allows for the
+secure execution of untrusted code within sandboxes, providing a safe environment for running potentially dangerous code.
 
 .. _Tutor: https://docs.tutor.overhang.io
-.. _service: https://github.com/eduNEXT/codejailservice
-.. _codejail: https://github.com/openedx/codejail
+.. _Codejail Service: https://github.com/eduNEXT/codejailservice
+.. _Codejail: https://github.com/openedx/codejail
 
 Installation
 ------------
 
+To install the latest version run:
+
 .. code-block:: bash
 
     pip install git+https://github.com/fgosew/tutor-contrib-codejail-sagemath.git
 
+You can install a specific version by adding the tag, branch, or commit:
+
+.. code-block:: bash
+
+    pip install git+https://github.com/edunext/[email protected]
+
 Usage
 -----
+Enable the plugin with:
 
 .. code-block:: bash
 
     tutor plugins enable codejail
 
-Then, you will have to install the "docker-edx-sandbox" apparmor profile on your host:
+Run the initialization jobs to install the required AppArmor profile on your host:
 
 .. code-block:: bash
 
@@ -46,23 +56,48 @@ Finally, the platform can be run as usual:
 
     tutor local launch
 
+**Please remember:** If the host is rebooted, the AppArmor profile needs to be reloaded.
+
 Configuration
 -------------
 
-For some of these configurations to work correctly, the codejail image must be built again. Command to build codejail: ``tutor images build codejail``.
+To customize the configuration, update the following settings in Tutor:
 
 - ``CODEJAIL_APPARMOR_DOCKER_IMAGE``: (default: ``docker.io/ednxops/codejail_apparmor_loader:latest``)
-- ``CODEJAIL_DOCKER_IMAGE``: (default: ``docker.io/ednxops/codejailservice:14.0.0``)
+- ``CODEJAIL_DOCKER_IMAGE``: (default: ``docker.io/ednxops/codejailservice:{{__version__}}``)
 - ``CODEJAIL_ENFORCE_APPARMOR`` (default: ``True``)
 - ``CODEJAIL_ENABLE_K8S_DAEMONSET`` (default: ``False``)
 - ``CODEJAIL_SKIP_INIT`` (default: ``False``)
 - ``CODEJAIL_SANDBOX_PYTHON_VERSION`` (default: ``3.8.6``)
 - ``CODEJAIL_EXTRA_PIP_REQUIREMENTS`` (optional) A list of pip requirements to add to your sandbox.
-    
-    .. code-block:: yaml
+- ``CODEJAIL_SERVICE_VERSION`` (default: ``release/redwood.1``),
+- ``CODEJAIL_SERVICE_REPOSITORY`` (default ``https://github.com/edunext/codejailservice.git```)
 
-        CODEJAIL_EXTRA_PIP_REQUIREMENTS:
-        - pybryt
+.. code-block:: yaml
+
+    CODEJAIL_EXTRA_PIP_REQUIREMENTS:
+    - pybryt
+
+
+Custom Image
+~~~~~~~~~~~~
+
+In most cases, you can work with the provided docker image for the release. However, you will need to re-build the docker image when:
+
+. Additional requirements are included in the sandbox via ``CODEJAIL_EXTRA_PIP_REQUIREMENTS``.
+- A different version of Python is set for the sandbox environment via ``CODEJAIL_SANDBOX_PYTHON_VERSION``.
+- The custom version of edx-platform that changes the contents of requirements/edx-sandbox.
+
+Create a new image running:
+
+.. code-block:: bash
+
+    # Add the tutor configuration with the custom value
+    tutor config save \
+    --set 'CODEJAIL_EXTRA_PIP_REQUIREMENTS=["pybryt"]'
+
+    # Build the image
+    tutor images build codejail
 
 
 Compatibility
@@ -83,68 +118,61 @@ Compatibility
 +------------------+---------------+
 | Quince           | >= 17.x       |
 +------------------+---------------+
+| Redwood          | >= 18.x       |
++------------------+---------------+
 
-**NOTE**: For the Open edx version of the Lilac release, the changes required for the Codejail service to interact with ``edx-platform`` are
-not included in ``open-release/lilac.master``. In order to use the service with the changes, please review `this PR`_.
+**NOTE**: For the Open edX version of the Lilac release, the changes required for the Codejail service to interact with ``edx-platform`` are
+not included in ``open-release/lilac.master``. To use the service with the changes, please review `this PR`_.
 
 .. _this PR: https://github.com/openedx/edx-platform/pull/27795
 
 Kubernetes Support
 ------------------
 
 The CodeJail service provides a sandbox to run arbitrary code. Security enforcement
-in the sandbox is done through AppArmor, this means that AppArmor must be installed
+in the sandbox is done through *AppArmor*, this means that AppArmor must be installed
 in the host machine and the `provided profile`_ must be loaded.
 
 .. _provided profile: tutorcodejail/templates/codejail/apps/profiles/docker-edx-sandbox
 
-The plugin provides an init task that runs a privileged container capable of loading
-the needed AppArmor profile unto your machine. This is only compatible with a docker
-installation. In Kubernetes you must guarantee that each node of your cluster has
-AppArmor installed and the profile loaded, for that reason the one time initialization
-task that is used in the init is skipped when running on kubernetes.
+The plugin provides an init task running a privileged container capable of loading the AppArmor profile onto your machine.
+This is only compatible with a docker installation.
 
-The plugins offers the possibility to load the AppArmor profile using a DaemonSet,
-assuming the nodes are already running AppArmor. To do so you must set
-``CODEJAIL_ENABLE_K8S_DAEMONSET`` to ``True``.
+For Kubernetes environments, ensure each node has AppArmor installed and the profile loaded. Optionally,
+set ``CODEJAIL_ENABLE_K8S_DAEMONSET`` to True to use a DaemonSet for loading the AppArmor profile,
+assuming the nodes are already running AppArmor.
 
-If, at your own discretion, want to run the service without enforcing the AppArmor
-profile you can set ``CODEJAIL_ENFORCE_APPARMOR`` to ``False``.
+If you choose to run the service without enforcing the AppArmor profile, you can set ``CODEJAIL_ENFORCE_APPARMOR`` to ``False``.
 
 More info about this discussion can be found on `this issue`_.
 
 .. _this issue: https://github.com/eduNEXT/tutor-contrib-codejail/issues/24
 
-Functionality test
-------------------
+Testing Functionality
+---------------------
 
-How to know if codejail is working
-~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
-
-The easiest way to test whether codejail is working is to validate it in ``Studio`` with a course
-that has loncapa problems.
-
-This test was performed on the Maple version of Open edx, using the course ``course_codejail_example.tar.gz``
-found in the additional resources section.
+To verify if Codejail is working, use a course with loncapa problems in ``Studio`` and check for correct execution.
+You can import the provided `example course`_.
 
 Once the course is imported, go to any section and select an exercise (`section example`_), the proper result is:
 
+.. _example course: https://github.com/eduNEXT/tutor-contrib-codejail/blob/main/docs/resources/course_codejail_example.tar.gz
 .. _section example: http://studio.local.overhang.io:8001/container/block-v1:edX+DemoX+Demo_Course+type@vertical+block@v-integral1
 
 .. image:: ./docs/resources/Codejailworking.png
     :width: 725px
     :align: center
     :alt: Example when codejail is working
 
-In this case, the section's content will render correctly and will be working as specified in the instructions of the problem.
+In this case, the section's content will render correctly and work as specified in the instructions of the problem.
 
 Possible failure case
 ~~~~~~~~~~~~~~~~~~~~~
 
-In case you forget to run ``tutor local do init --limit codejail`` for apparmor profile, this error in
+In case you forget to run ``tutor local do init --limit codejail`` for AppArmor profile, this error in
 ``Studio`` will arise::
 
-    Error formatting HTML for problem:
+    Error formatting HTML for the problem:
     cannot create LoncapaProblem block-v1:edX+DemoX+Demo_Course+type@problem+block@integral1: Error while
     executing script code: Codejail API Service is unavailable. Please try again in a few minutes.
 
@@ -153,15 +181,8 @@ In case you forget to run ``tutor local do init --limit codejail`` for apparmor
     :align: center
     :alt: Example when codejail is not working
 
-This indicates that the codejail service is not turned on or is not working properly. Be sure to follow the
-steps in the usage section so this doesn't happen.
-
-Additional Resources
---------------------
-
-Example course to test the Codejail service: `course_codejail_example.tar.gz`_
-
-.. _course_codejail_example.tar.gz: https://github.com/eduNEXT/tutor-contrib-codejail/blob/main/docs/resources/course_codejail_example.tar.gz
+This indicates that the Codejail service is either not turned on or not working properly. Please ensure to follow
+the steps outlined in the usage section to prevent this issue.
 
 How to Contribute
 -----------------
@@ -175,4 +196,4 @@ quality, which will make your contribution more likely to be accepted.
 License
 -------
 
-This software is licensed under the terms of the AGPLv3.
+This software is licensed under the terms of the AGPLv3. See the LICENSE file for details.

setup.py

@@ -44,7 +44,7 @@ def load_about():
     packages=find_packages(exclude=["tests*"]),
     include_package_data=True,
     python_requires=">=3.8",
-    install_requires=["tutor>=17.0.2, <18"],
+    install_requires=["tutor>=18.0.0, <19"],
     entry_points={"tutor.plugin.v1": ["codejail = tutorcodejail.plugin"]},
     classifiers=[
         "Development Status :: 3 - Alpha",
@@ -55,5 +55,7 @@ def load_about():
         "Programming Language :: Python :: 3.8",
         "Programming Language :: Python :: 3.9",
         "Programming Language :: Python :: 3.10",
+        "Programming Language :: Python :: 3.11",
+        "Programming Language :: Python :: 3.12",
     ],
 )

tutorcodejail/__about__.py

@@ -1,2 +1,2 @@
 """Helps you keep your cool when creating dozens of open edX and eduNEXT environments."""
-__version__ = "17.0.2"
+__version__ = "18.0.0"

tutorcodejail/plugin.py

@@ -20,7 +20,7 @@
         "ENABLE_K8S_DAEMONSET": False,
         "ENFORCE_APPARMOR": True,
         "HOST": "codejailservice",
-        "SANDBOX_PYTHON_VERSION": "3.8.6",
+        "SANDBOX_PYTHON_VERSION": "3.11.9",
         "SKIP_INIT": False,
         "LIMIT_CPU": "1",
         "LIMIT_MEMORY": "1Gi",
@@ -30,6 +30,8 @@
         "MIN_REPLICAS": 1,
         "MAX_REPLICAS": 4,
         "AVG_CPU": 65,
+        "SERVICE_VERSION": "release/redwood.1",
+        "SERVICE_REPOSITORY": "https://github.com/edunext/codejailservice.git",
     },
     "overrides": {},
 }

tutorcodejail/templates/codejail/build/codejail/Dockerfile

@@ -50,6 +50,17 @@ ENV HOME /root
 WORKDIR /root
 ENV PYENV_ROOT /opt/pyenv
 
+#<<<<<<< production
+#RUN git clone https://github.com/pyenv/pyenv $PYENV_ROOT --branch v2.4.0 --depth 1
+
+#ARG CODEJAILSERVICE_PYTHON_VERSION=3.11.9
+#RUN $PYENV_ROOT/bin/pyenv install $CODEJAILSERVICE_PYTHON_VERSION
+
+#ARG SANDBOX_PYTHON_VERSION={{ CODEJAIL_SANDBOX_PYTHON_VERSION }}
+#RUN git clone https://github.com/esinker/pyenv-version-alias $PYENV_ROOT/plugins/pyenv-alias
+#RUN VERSION_ALIAS={{ CODEJAIL_SANDBOX_PYTHON_VERSION }}_sandbox $PYENV_ROOT/bin/pyenv install -f $SANDBOX_PYTHON_VERSION
+#=======
+
 #needed or the https request / tls in general will fail
 RUN update-ca-certificates
 
@@ -80,6 +91,7 @@ ARG CODEJAILSERVICE_PYTHON_VERSION=3.8.6
 
 RUN . "${HOME}/.bash_profile" && \
     pyenv install "${CODEJAILSERVICE_PYTHON_VERSION}" 
+#>>>>>>> main
 
 RUN $PYENV_ROOT/versions/$CODEJAILSERVICE_PYTHON_VERSION/bin/python -m venv /openedx/venv
 #we don't install conda here yet as this would waste space due to the stages being reused later on.
@@ -89,8 +101,12 @@ RUN $PYENV_ROOT/versions/$CODEJAILSERVICE_PYTHON_VERSION/bin/python -m venv /ope
 ###### Codejail service code
 
 FROM minimal as code
+#<<<<<<< production
+#RUN git clone {{ CODEJAIL_SERVICE_REPOSITORY }} --branch {{ CODEJAIL_SERVICE_VERSION }} --depth 1 /openedx/codejailservice
+#=======
 
 RUN git clone https://github.com/eduNEXT/codejailservice.git --branch {{ CODEJAIL_VERSION }} --depth 1 /openedx/codejailservice
+#>>>>>>> main
 WORKDIR /openedx/codejailservice
 
 ###### Install python requirements in virtualenv
@@ -117,8 +133,8 @@ ENV VIRTUAL_ENV /sandbox/venv/
 WORKDIR /var/tmp
 RUN mkdir -p common/lib/
 
-COPY --from={{ DOCKER_IMAGE_OPENEDX }} /openedx/edx-platform/requirements/edx-sandbox/py38.txt py38.txt
-RUN pip3 install -r py38.txt
+COPY --from={{ DOCKER_IMAGE_OPENEDX }} /openedx/edx-platform/requirements/edx-sandbox/releases/redwood.txt redwood.txt
+RUN pip3 install -r redwood.txt
 
 ##### Prod image
 FROM minimal as production