diff --git a/CHANGELOG.md b/CHANGELOG.md index 34f5d467ea7..1fb3cb51501 100644 --- a/CHANGELOG.md +++ b/CHANGELOG.md @@ -20,6 +20,18 @@ instructions, because git commits are used to generate release notes: + +## v17.0.0 (2023-12-09) + +- 💥[Feature] Upgrade to Quince. (by @regisb) +- 💥[Feature] Replace "*.local.overhang.io" hostnames by "*.local.edly.io". (by @regisb) +- 💥[Feature] Enable the Indigo theme by default, if no other theme is set. (by @regisb) +- 💥[Deprecation] Tutor no longer supports the legacy Docker builder, which was previously available by setting `DOCKER_BUILDKIT=0` in the host environment. Going forward, Tutor will always use BuildKit (a.k.a. `docker buildx` in Docker v19-v22, or just `docker build` in Docker v23). This transition will improve build performance and should be seamless for Tutor users who are running a supported Docker version (by @kdmccormick). +- 💥[Deprecation] The template variable `is_buildkit_enabled`, which now always returns True, is deprecated. Plugin authors should assume BuildKit is enabled and remove the variable from their templates (by @kdmccormick). +- 💥[Deprecation] Adding Python packages to edx-platform via `private.txt` is no longer supported. Instead, users should bind-mount their requirement directories with `tutor mounts add ...`. (by @regisb) +- [Bugfix] Updated how the Tutor setting `JWT_RSA_PRIVATE_KEY` is rendered into the LMS Django setting `JWT_AUTH['JWT_PRIVATE_SIGNING_JWK']` as required by a recent breaking upstream change. The new representation of the `JWT_PRIVATE_SIGNING_JWK` simply adds the `dq`, `dp`, and `qi` parameters. Without this fix, LMS would encounter an `InvalidKeyError` on all logins. (by @kdmccormick) +- [Improvement] You don't have to run `tutor config save` every time you enable or disable a plugin anymore. (by @CodeWithEmad) + ## v16.1.8 (2023-12-10) diff --git a/changelog.d/20230818_112124_kyle_buildkit.md b/changelog.d/20230818_112124_kyle_buildkit.md deleted file mode 100644 index 7084b290415..00000000000 --- a/changelog.d/20230818_112124_kyle_buildkit.md +++ /dev/null @@ -1,2 +0,0 @@ -- [Deprecation] The template variable ``is_buildkit_enabled``, which now always returns True, is deprecated. Plugin authors should assume BuildKit is enabled and remove the variable from their templates (by @kdmccormick). -- 💥[Deprecation] Tutor no longer supports the legacy Docker builder, which was previously available by setting ``DOCKER_BUILDKIT=0`` in the host environment. Going forward, Tutor will always use BuildKit (a.k.a. ``docker buildx`` in Docker v19-v22, or just ``docker build`` in Docker v23). This transition will improve build performance and should be seamless for Tutor users who are running a supported Docker version (by @kdmccormick). diff --git a/changelog.d/20231011_000434_codewithemad_save_configs.md b/changelog.d/20231011_000434_codewithemad_save_configs.md deleted file mode 100644 index 0afbad42ead..00000000000 --- a/changelog.d/20231011_000434_codewithemad_save_configs.md +++ /dev/null @@ -1 +0,0 @@ -- [Improvement] You don't have to run `tutor config save` every time you enable or disable a plugin anymore. (by @CodeWithEmad) \ No newline at end of file diff --git a/changelog.d/20231020_143112_kyle_jwk_full_complement.md b/changelog.d/20231020_143112_kyle_jwk_full_complement.md deleted file mode 100644 index 162b89db405..00000000000 --- a/changelog.d/20231020_143112_kyle_jwk_full_complement.md +++ /dev/null @@ -1 +0,0 @@ -- [Bugfix] Updated how the Tutor setting ``JWT_RSA_PRIVATE_KEY`` is rendered into the LMS Django setting ``JWT_AUTH['JWT_PRIVATE_SIGNING_JWK']`` as required by a recent breaking upstream change. The new representation of the ``JWT_PRIVATE_SIGNING_JWK`` simply adds the ``dq``, ``dp``, and ``qi`` parameters. Without this fix, LMS would encounter an ``InvalidKeyError`` on all logins (by @kdmccormick). diff --git a/changelog.d/20231205_113430_regis_local_edly_io.md b/changelog.d/20231205_113430_regis_local_edly_io.md deleted file mode 100644 index dec279fb6b6..00000000000 --- a/changelog.d/20231205_113430_regis_local_edly_io.md +++ /dev/null @@ -1 +0,0 @@ -- 💥[Feature] Replace "*.local.overhang.io" hostnames by "*.local.edly.io". (by @regisb) diff --git a/docs/configuration.rst b/docs/configuration.rst index 2efd18054f8..0f88d79e198 100644 --- a/docs/configuration.rst +++ b/docs/configuration.rst @@ -136,7 +136,7 @@ Open edX customisation This defines the git repository from which you install Open edX platform code. If you run an Open edX fork with custom patches, set this to your own git repository. You may also override this configuration parameter at build time, by providing a ``--build-arg`` option. -- ``OPENEDX_COMMON_VERSION`` (default: ``"open-release/palm.4"``, or ``master`` in :ref:`nightly `) +- ``OPENEDX_COMMON_VERSION`` (default: ``"open-release/quince.1"``, or ``master`` in :ref:`nightly `) This defines the default version that will be pulled from all Open edX git repositories. @@ -156,9 +156,9 @@ These two configuration parameters define which Redis database to use for Open e .. _openedx_extra_pip_requirements: -- ``OPENEDX_EXTRA_PIP_REQUIREMENTS`` (default: ``["openedx-scorm-xblock>=16.0.0,<17.0.0"]``) +- ``OPENEDX_EXTRA_PIP_REQUIREMENTS`` (default: ``[]``) -This defines extra pip packages that are going to be installed for Open edX. +Define extra pip packages that are going to be installed for edx-platform. - ``NPM_REGISTRY`` (default: ``"https://registry.npmjs.org/"``) @@ -354,36 +354,16 @@ See :ref:`the corresponding tutorial `. .. _custom_extra_xblocks: -Installing extra xblocks and requirements ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ -Would you like to include custom xblocks, or extra requirements to your Open edX platform? Additional requirements can be added to the ``OPENEDX_EXTRA_PIP_REQUIREMENTS`` parameter in the :ref:`config file ` or to the ``env/build/openedx/requirements/private.txt`` file. The difference between them, is that ``private.txt`` file, even though it could be used for both, :ref:`should be used for installing extra xblocks or requirements from private repositories `. For instance, to include the `polling xblock from Opencraft `_: +Would you like to include custom xblocks, or extra requirements to your Open edX platform? Additional requirements can be added to the ``OPENEDX_EXTRA_PIP_REQUIREMENTS`` parameter in the :ref:`config file `. For instance, to include the `polling xblock from Opencraft `_: tutor config save --append OPENEDX_EXTRA_PIP_REQUIREMENTS=git+https://github.com/open-craft/xblock-poll.git -Alternatively, add the dependency to ``private.txt``:: - - echo "git+https://github.com/open-craft/xblock-poll.git" >> "$(tutor config printroot)/env/build/openedx/requirements/private.txt" - Then, the ``openedx`` docker image must be rebuilt:: tutor images build openedx -.. _extra_private_xblocks: - -Installing extra requirements from private repositories -******************************************************* - -When installing extra xblock or requirements from private repositories, ``private.txt`` file should be used, because it allows installing dependencies without adding git credentials to the Docker image. By adding your git credentials to the Docker image, you're risking leaking your git credentials, if you were to publish (intentionally or unintentionally) the Docker image in a public place. - -To install xblocks from a private repository that requires authentication, you must first clone the repository inside the ``openedx/requirements`` folder on the host:: - - git clone git@github.com:me/myprivaterepo.git "$(tutor config printroot)/env/build/openedx/requirements/myprivaterepo" - -Then, declare your extra requirements with the ``-e`` flag in ``openedx/requirements/private.txt``:: - - echo "-e ./myprivaterepo" >> "$(tutor config printroot)/env/build/openedx/requirements/private.txt" - .. _edx_platform_fork: Running a fork of ``edx-platform`` @@ -401,16 +381,16 @@ Note that your edx-platform version must be a fork of the latest release **tag** If you don't create your fork from this tag, you *will* have important compatibility issues with other services. In particular: -- Do not try to run a fork from an older (pre-Palm) version of edx-platform: this will simply not work. +- Do not try to run a fork from an older (pre-Quince) version of edx-platform: this will simply not work. - Do not try to run a fork from the edx-platform master branch: there is a 99% probability that it will fail. -- Do not try to run a fork from the open-release/palm.master branch: Tutor will attempt to apply security and bug fix patches that might already be included in the open-release/palm.master but which were not yet applied to the latest release tag. Patch application will thus fail if you base your fork from the open-release/palm.master branch. +- Do not try to run a fork from the open-release/quince.master branch: Tutor will attempt to apply security and bug fix patches that might already be included in the open-release/quince.master but which were not yet applied to the latest release tag. Patch application will thus fail if you base your fork from the open-release/quince.master branch. .. _i18n: Adding custom translations ~~~~~~~~~~~~~~~~~~~~~~~~~~ -If you are not running Open edX in English (``LANGUAGE_CODE`` default: ``"en"``), chances are that some strings will not be properly translated. In most cases, this is because not enough contributors have helped translate Open edX into your language. It happens! With Tutor, available translated languages include those that come bundled with `edx-platform `__ as well as those from `openedx-i18n `__. +If you are not running Open edX in English (``LANGUAGE_CODE`` default: ``"en"``), chances are that some strings will not be properly translated. In most cases, this is because not enough contributors have helped translate Open edX into your language. It happens! With Tutor, available translated languages include those that come bundled with `edx-platform `__ as well as those from `openedx-i18n `__. Tutor offers a relatively simple mechanism to add custom translations to the openedx Docker image. You should create a folder that corresponds to your language code in the "build/openedx/locale" folder of the Tutor environment. This folder should contain a "LC_MESSAGES" folder. For instance:: @@ -431,9 +411,9 @@ Then, add a "django.po" file there that will contain your custom translations:: .. warning:: Don't forget to specify the file ``Content-Type`` when adding message strings with non-ASCII characters; otherwise a ``UnicodeDecodeError`` will be raised during compilation. -The "String to translate" part should match *exactly* the string that you would like to translate. You cannot make it up! The best way to find this string is to copy-paste it from the `upstream django.po file for the English language `__. +The "String to translate" part should match *exactly* the string that you would like to translate. You cannot make it up! The best way to find this string is to copy-paste it from the `upstream django.po file for the English language `__. -If you cannot find the string to translate in this file, then it means that you are trying to translate a string that is used in some piece of javascript code. Those strings are stored in a different file named "djangojs.po". You can check it out `in the edx-platform repo as well `__. Your custom javascript strings should also be stored in a "djangojs.po" file that should be placed in the same directory. +If you cannot find the string to translate in this file, then it means that you are trying to translate a string that is used in some piece of javascript code. Those strings are stored in a different file named "djangojs.po". You can check it out `in the edx-platform repo as well `__. Your custom javascript strings should also be stored in a "djangojs.po" file that should be placed in the same directory. To recap, here is an example. To translate a few strings in French, both from django.po and djangojs.po, we would have the following file hierarchy:: diff --git a/docs/dev.rst b/docs/dev.rst index 278975e4f6d..6b1f1975bf5 100644 --- a/docs/dev.rst +++ b/docs/dev.rst @@ -113,7 +113,7 @@ The ``openedx-dev`` Docker image is based on the same ``openedx`` image used by - The user that runs inside the container has the same UID as the user on the host, to avoid permission problems inside mounted volumes (and in particular in the edx-platform repository). - Additional Python and system requirements are installed for convenient debugging: `ipython `__, `ipdb `__, vim, telnet. -- The edx-platform `development requirements `__ are installed. +- The edx-platform `development requirements `__ are installed. If you are using a custom ``openedx`` image, then you will need to rebuild ``openedx-dev`` every time you modify ``openedx``. To so, run:: diff --git a/docs/faq.rst b/docs/faq.rst index f4907d854c8..6467ecc4ac9 100644 --- a/docs/faq.rst +++ b/docs/faq.rst @@ -38,7 +38,7 @@ The `devstack `_ is meant for development o Is Tutor officially supported by edX? ------------------------------------- -Yes: as of the Open edX Maple release (December 9th 2021), Tutor is the only officially supported installation method for Open edX: see the `official installation instructions `__. +Yes: as of the Open edX Maple release (December 9th 2021), Tutor is the only officially supported installation method for Open edX: see the `official installation instructions `__. What features are missing from Tutor? ------------------------------------- diff --git a/docs/install.rst b/docs/install.rst index df4069261c2..b058ddced58 100644 --- a/docs/install.rst +++ b/docs/install.rst @@ -12,7 +12,7 @@ Requirements * Architecture: Both AMD64 and ARM64 are supported. * Required software: - - `Docker `__: v20.10.15+ + - `Docker `__: v24.0.5+ (with BuildKit 0.11+) - `Docker Compose `__: v2.0.0+ .. warning:: @@ -123,11 +123,11 @@ Major Open edX releases are published twice a year, in June and December, by the 4. Test the new release in a sandboxed environment. 5. If you are running edx-platform, or some other repository from a custom branch, then you should rebase (and test) your changes on top of the latest release tag (see :ref:`edx_platform_fork`). -The process for upgrading from one major release to the next works similarly to any other upgrade, with the ``launch`` command (see above). The single difference is that if the ``launch`` command detects that your tutor environment was generated with an older release, it will perform a few release-specific upgrade steps. These extra upgrade steps will be performed just once. But they will be ignored if you updated your local environment (for instance: with ``tutor config save``) before running ``launch``. This situation typically occurs if you need to re-build some Docker images (see above). In such a case, you should make use of the ``upgrade`` command. For instance, to upgrade a local installation from Olive to Palm and rebuild some Docker images, run:: +The process for upgrading from one major release to the next works similarly to any other upgrade, with the ``launch`` command (see above). The single difference is that if the ``launch`` command detects that your tutor environment was generated with an older release, it will perform a few release-specific upgrade steps. These extra upgrade steps will be performed just once. But they will be ignored if you updated your local environment (for instance: with ``tutor config save``) before running ``launch``. This situation typically occurs if you need to re-build some Docker images (see above). In such a case, you should make use of the ``upgrade`` command. For instance, to upgrade a local installation from Palm to Quince and rebuild some Docker images, run:: tutor config save tutor images build all # list the images that should be rebuilt here - tutor local upgrade --from=olive + tutor local upgrade --from=palm tutor local launch .. _autocomplete: diff --git a/docs/quickstart.rst b/docs/quickstart.rst index cad6dd20773..58cf4d4e32f 100644 --- a/docs/quickstart.rst +++ b/docs/quickstart.rst @@ -22,7 +22,7 @@ Yes :) This is what happens when you run ``tutor local launch``: 2. Configuration files are generated from templates. 3. Docker images are downloaded. 4. Docker containers are provisioned. -5. A full, production-ready Open edX platform (`Palm `__ release) is run with docker-compose. +5. A full, production-ready Open edX platform (`Quince `__ release) is run with docker-compose. The whole procedure should require less than 10 minutes, on a server with good bandwidth. Note that your host environment will not be affected in any way, since everything runs inside docker containers. Root access is not even necessary. diff --git a/docs/reference/indexes.rst b/docs/reference/indexes.rst index db06fcfadb6..e9c337ea337 100644 --- a/docs/reference/indexes.rst +++ b/docs/reference/indexes.rst @@ -7,10 +7,10 @@ Plugin indexes are a great way to have your plugins discovered by other users. P Index file paths ================ -A plugin index is a yaml-formatted file. It can be stored on the web or on your computer. In both cases, the index file location must end with "/plugins.yml". For instance, the following are valid index locations if you run the Open edX "Palm" release: +A plugin index is a yaml-formatted file. It can be stored on the web or on your computer. In both cases, the index file location must end with "/plugins.yml". For instance, the following are valid index locations if you run the Open edX "Quince" release: -- https://overhang.io/tutor/main/palm/plugins.yml -- ``/path/to/your/local/index/palm/plugins.yml`` +- https://overhang.io/tutor/main/quince/plugins.yml +- ``/path/to/your/local/index/quince/plugins.yml`` To add either indexes, run the ``tutor plugins index add`` command without the suffix. For instance:: @@ -106,9 +106,9 @@ Manage plugins in development Plugin developers and maintainers often want to install local versions of their plugins. They usually achieve this with ``pip install -e /path/to/tutor-plugin``. We can improve that workflow by creating an index for local plugins:: # Create the plugin index directory - mkdir -p ~/localindex/palm/ + mkdir -p ~/localindex/quince/ # Edit the index - vim ~/localindex/palm/plugins.yml + vim ~/localindex/quince/plugins.yml Add the following to the index:: diff --git a/docs/tutorials/edx-platform.rst b/docs/tutorials/edx-platform.rst index fde1ea97370..7d60e6b74d7 100644 --- a/docs/tutorials/edx-platform.rst +++ b/docs/tutorials/edx-platform.rst @@ -69,7 +69,7 @@ Quite often, developers don't want to work on edx-platform directly, but on a de cd /my/workspace/edx-ora2 git clone https://github.com/openedx/edx-ora2 . -Then, check out the right version of the package. This is the version that is indicated in the `edx-platform/requirements/edx/base.txt `__. Be careful that the version that is currently in use in your version of edx-platform is **not necessarily the head of the master branch**:: +Then, check out the right version of the package. This is the version that is indicated in the `edx-platform/requirements/edx/base.txt `__. Be careful that the version that is currently in use in your version of edx-platform is **not necessarily the head of the master branch**:: git checkout diff --git a/requirements/base.txt b/requirements/base.txt index e79b5bdcc25..53ea28465eb 100644 --- a/requirements/base.txt +++ b/requirements/base.txt @@ -6,27 +6,27 @@ # appdirs==1.4.4 # via -r requirements/base.in -cachetools==5.3.1 +cachetools==5.3.2 # via google-auth certifi==2023.7.22 # via # kubernetes # requests -charset-normalizer==3.2.0 +charset-normalizer==3.3.1 # via requests click==8.1.7 # via -r requirements/base.in -google-auth==2.22.0 +google-auth==2.23.3 # via kubernetes idna==3.4 # via requests jinja2==3.1.2 # via -r requirements/base.in -kubernetes==27.2.0 +kubernetes==28.1.0 # via -r requirements/base.in markupsafe==2.1.3 # via jinja2 -mypy==1.5.1 +mypy==1.6.1 # via -r requirements/base.in mypy-extensions==1.0.0 # via mypy @@ -40,7 +40,7 @@ pyasn1==0.5.0 # rsa pyasn1-modules==0.3.0 # via google-auth -pycryptodome==3.18.0 +pycryptodome==3.19.0 # via -r requirements/base.in python-dateutil==2.8.2 # via kubernetes @@ -58,19 +58,17 @@ rsa==4.9 # via google-auth six==1.16.0 # via - # google-auth # kubernetes # python-dateutil tomli==2.0.1 # via mypy -typing-extensions==4.7.1 +typing-extensions==4.8.0 # via # -r requirements/base.in # mypy urllib3==1.26.18 # via - # google-auth # kubernetes # requests -websocket-client==1.6.2 +websocket-client==1.6.4 # via kubernetes diff --git a/requirements/dev.in b/requirements/dev.in index c53771c4193..ef0d7ada7b6 100644 --- a/requirements/dev.in +++ b/requirements/dev.in @@ -7,6 +7,12 @@ pyinstaller scriv twine +# Constraints +# for compatibility with sphinx-rtd-theme +# drop this constraint once sphinx-rtd-theme 2.0.0 comes out +# https://github.com/readthedocs/sphinx_rtd_theme/tags +docutils<0.19.0 + # Types packages types-docutils types-PyYAML diff --git a/requirements/dev.txt b/requirements/dev.txt index 8d8b265839a..fd7ba3e7479 100644 --- a/requirements/dev.txt +++ b/requirements/dev.txt @@ -4,21 +4,19 @@ # # pip-compile requirements/dev.in # -altgraph==0.17.3 +altgraph==0.17.4 # via pyinstaller appdirs==1.4.4 # via -r requirements/base.txt -astroid==2.15.6 +astroid==3.0.1 # via pylint attrs==23.1.0 # via scriv -black==23.7.0 +black==23.10.1 # via -r requirements/dev.in -bleach==6.0.0 - # via readme-renderer -build==0.10.0 +build==1.0.3 # via pip-tools -cachetools==5.3.1 +cachetools==5.3.2 # via # -r requirements/base.txt # google-auth @@ -27,9 +25,9 @@ certifi==2023.7.22 # -r requirements/base.txt # kubernetes # requests -cffi==1.15.1 +cffi==1.16.0 # via cryptography -charset-normalizer==3.2.0 +charset-normalizer==3.3.1 # via # -r requirements/base.txt # requests @@ -42,15 +40,17 @@ click==8.1.7 # scriv click-log==0.4.0 # via scriv -coverage==7.3.0 +coverage==7.3.2 # via -r requirements/dev.in cryptography==41.0.7 # via secretstorage dill==0.3.7 # via pylint -docutils==0.20.1 - # via readme-renderer -google-auth==2.22.0 +docutils==0.18.1 + # via + # -r requirements/dev.in + # readme-renderer +google-auth==2.23.3 # via # -r requirements/base.txt # kubernetes @@ -60,7 +60,9 @@ idna==3.4 # requests importlib-metadata==6.8.0 # via + # build # keyring + # pyinstaller # twine importlib-resources==6.1.1 # via keyring @@ -78,12 +80,12 @@ jinja2==3.1.2 # scriv keyring==24.2.0 # via twine -kubernetes==27.2.0 +kubernetes==28.1.0 # via -r requirements/base.txt -lazy-object-proxy==1.9.0 - # via astroid markdown-it-py==3.0.0 - # via rich + # via + # rich + # scriv markupsafe==2.1.3 # via # -r requirements/base.txt @@ -94,29 +96,32 @@ mdurl==0.1.2 # via markdown-it-py more-itertools==10.1.0 # via jaraco-classes -mypy==1.5.1 +mypy==1.6.1 # via -r requirements/base.txt mypy-extensions==1.0.0 # via # -r requirements/base.txt # black # mypy +nh3==0.2.14 + # via readme-renderer oauthlib==3.2.2 # via # -r requirements/base.txt # kubernetes # requests-oauthlib -packaging==23.1 +packaging==23.2 # via # black # build + # pyinstaller pathspec==0.11.2 # via black pip-tools==7.3.0 # via -r requirements/dev.in pkginfo==1.9.6 # via twine -platformdirs==3.10.0 +platformdirs==3.11.0 # via # black # pylint @@ -131,17 +136,17 @@ pyasn1-modules==0.3.0 # google-auth pycparser==2.21 # via cffi -pycryptodome==3.18.0 +pycryptodome==3.19.0 # via -r requirements/base.txt pygments==2.16.1 # via # readme-renderer # rich -pyinstaller==5.13.1 +pyinstaller==6.1.0 # via -r requirements/dev.in -pyinstaller-hooks-contrib==2023.7 +pyinstaller-hooks-contrib==2023.10 # via pyinstaller -pylint==2.17.5 +pylint==3.0.2 # via -r requirements/dev.in pyproject-hooks==1.0.0 # via build @@ -153,7 +158,7 @@ pyyaml==6.0.1 # via # -r requirements/base.txt # kubernetes -readme-renderer==41.0 +readme-renderer==42.0 # via twine requests==2.31.0 # via @@ -171,21 +176,19 @@ requests-toolbelt==1.0.0 # via twine rfc3986==2.0.0 # via twine -rich==13.5.2 +rich==13.6.0 # via twine rsa==4.9 # via # -r requirements/base.txt # google-auth -scriv==1.3.1 +scriv==1.5.0 # via -r requirements/dev.in secretstorage==3.3.3 # via keyring six==1.16.0 # via # -r requirements/base.txt - # bleach - # google-auth # kubernetes # python-dateutil tomli==2.0.1 @@ -203,11 +206,11 @@ twine==4.0.2 # via -r requirements/dev.in types-docutils==0.20.0.3 # via -r requirements/dev.in -types-pyyaml==6.0.12.11 +types-pyyaml==6.0.12.12 # via -r requirements/dev.in -types-setuptools==68.1.0.0 +types-setuptools==68.2.0.0 # via -r requirements/dev.in -typing-extensions==4.7.1 +typing-extensions==4.8.0 # via # -r requirements/base.txt # astroid @@ -218,21 +221,16 @@ typing-extensions==4.7.1 urllib3==1.26.18 # via # -r requirements/base.txt - # google-auth # kubernetes # requests # twine -webencodings==0.5.1 - # via bleach -websocket-client==1.6.2 +websocket-client==1.6.4 # via # -r requirements/base.txt # kubernetes wheel==0.41.2 # via pip-tools -wrapt==1.15.0 - # via astroid -zipp==3.16.2 +zipp==3.17.0 # via # importlib-metadata # importlib-resources diff --git a/requirements/docs.txt b/requirements/docs.txt index 80bb0301101..c4f5c6e9c2d 100644 --- a/requirements/docs.txt +++ b/requirements/docs.txt @@ -8,9 +8,9 @@ alabaster==0.7.13 # via sphinx appdirs==1.4.4 # via -r requirements/base.txt -babel==2.12.1 +babel==2.13.1 # via sphinx -cachetools==5.3.1 +cachetools==5.3.2 # via # -r requirements/base.txt # google-auth @@ -19,7 +19,7 @@ certifi==2023.7.22 # -r requirements/base.txt # kubernetes # requests -charset-normalizer==3.2.0 +charset-normalizer==3.3.1 # via # -r requirements/base.txt # requests @@ -32,7 +32,7 @@ docutils==0.18.1 # sphinx # sphinx-click # sphinx-rtd-theme -google-auth==2.22.0 +google-auth==2.23.3 # via # -r requirements/base.txt # kubernetes @@ -48,13 +48,13 @@ jinja2==3.1.2 # via # -r requirements/base.txt # sphinx -kubernetes==27.2.0 +kubernetes==28.1.0 # via -r requirements/base.txt markupsafe==2.1.3 # via # -r requirements/base.txt # jinja2 -mypy==1.5.1 +mypy==1.6.1 # via -r requirements/base.txt mypy-extensions==1.0.0 # via @@ -65,7 +65,7 @@ oauthlib==3.2.2 # -r requirements/base.txt # kubernetes # requests-oauthlib -packaging==23.1 +packaging==23.2 # via sphinx pyasn1==0.5.0 # via @@ -76,7 +76,7 @@ pyasn1-modules==0.3.0 # via # -r requirements/base.txt # google-auth -pycryptodome==3.18.0 +pycryptodome==3.19.0 # via -r requirements/base.txt pygments==2.16.1 # via sphinx @@ -107,7 +107,6 @@ rsa==4.9 six==1.16.0 # via # -r requirements/base.txt - # google-auth # kubernetes # python-dateutil snowballstemmer==2.2.0 @@ -140,17 +139,16 @@ tomli==2.0.1 # via # -r requirements/base.txt # mypy -typing-extensions==4.7.1 +typing-extensions==4.8.0 # via # -r requirements/base.txt # mypy urllib3==1.26.18 # via # -r requirements/base.txt - # google-auth # kubernetes # requests -websocket-client==1.6.2 +websocket-client==1.6.4 # via # -r requirements/base.txt # kubernetes diff --git a/requirements/plugins.txt b/requirements/plugins.txt index e67affd113c..c78ef5773fb 100644 --- a/requirements/plugins.txt +++ b/requirements/plugins.txt @@ -1,12 +1,13 @@ -# change version ranges when upgrading from palm -tutor-android>=16.0.0,<17.0.0 -tutor-cairn>=16.0.0,<17.0.0 -tutor-discovery>=16.0.0,<17.0.0 -tutor-ecommerce>=16.0.0,<17.0.0 -tutor-forum>=16.0.0,<17.0.0 -tutor-license>=16.0.0,<17.0.0 -tutor-mfe>=16.0.0,<17.0.0 -tutor-minio>=16.0.0,<17.0.0 -tutor-notes>=16.0.0,<17.0.0 -tutor-webui>=16.0.0,<17.0.0 -tutor-xqueue>=16.0.0,<17.0.0 +# change version ranges when upgrading from quince +tutor-android>=17.0.0,<18.0.0 +tutor-cairn>=17.0.0,<18.0.0 +tutor-credentials>=17.0.0,<18.0.0 +tutor-discovery>=17.0.0,<18.0.0 +tutor-ecommerce>=17.0.0,<18.0.0 +tutor-forum>=17.0.0,<18.0.0 +tutor-jupyter>=17.0.0,<18.0.0 +tutor-mfe>=17.0.0,<18.0.0 +tutor-minio>=17.0.0,<18.0.0 +tutor-notes>=17.0.0,<18.0.0 +tutor-webui>=17.0.0,<18.0.0 +tutor-xqueue>=17.0.0,<18.0.0 diff --git a/tests/commands/test_config.py b/tests/commands/test_config.py index 5c9dcea307a..58b15edd93a 100644 --- a/tests/commands/test_config.py +++ b/tests/commands/test_config.py @@ -94,9 +94,8 @@ def test_config_append_with_existing_default(self) -> None: ) config = tutor_config.load(root) assert isinstance(config["OPENEDX_EXTRA_PIP_REQUIREMENTS"], list) - self.assertEqual(2, len(config["OPENEDX_EXTRA_PIP_REQUIREMENTS"])) self.assertEqual( - "my-package==1.0.0", config["OPENEDX_EXTRA_PIP_REQUIREMENTS"][1] + ["my-package==1.0.0"], config["OPENEDX_EXTRA_PIP_REQUIREMENTS"] ) diff --git a/tests/test_env.py b/tests/test_env.py index a79060991a3..27cff9678bd 100644 --- a/tests/test_env.py +++ b/tests/test_env.py @@ -260,7 +260,7 @@ def test_current_version_in_latest_env(self) -> None: ) as f: f.write(__version__) self.assertEqual(__version__, env.current_version(root)) - self.assertEqual("palm", env.get_env_release(root)) + self.assertEqual("quince", env.get_env_release(root)) self.assertIsNone(env.should_upgrade_from_release(root)) self.assertTrue(env.is_up_to_date(root)) diff --git a/tutor/__about__.py b/tutor/__about__.py index 2599ad87d1c..33297dd3a4d 100644 --- a/tutor/__about__.py +++ b/tutor/__about__.py @@ -2,7 +2,7 @@ # Increment this version number to trigger a new release. See # docs/tutor.html#versioning for information on the versioning scheme. -__version__ = "16.1.8" +__version__ = "17.0.0" # The version suffix will be appended to the actual version, separated by a # dash. Use this suffix to differentiate between the actual released version and @@ -10,7 +10,7 @@ # the nightly branch. # The suffix is cleanly separated from the __version__ in this module to avoid # conflicts when merging branches. -__version_suffix__ = "nightly" +__version_suffix__ = "" # The app name will be used to define the name of the default tutor root and # plugin directory. To avoid conflicts between multiple locally-installed diff --git a/tutor/commands/upgrade/__init__.py b/tutor/commands/upgrade/__init__.py index 9c1d2034249..b7a4e028dec 100644 --- a/tutor/commands/upgrade/__init__.py +++ b/tutor/commands/upgrade/__init__.py @@ -1,4 +1,4 @@ -# Note: don't forget to change this when we upgrade from palm +# Note: don't forget to change this when we upgrade from quince OPENEDX_RELEASE_NAMES = [ "ironwood", "juniper", @@ -7,4 +7,6 @@ "maple", "nutmeg", "olive", + "palm", + "quince", ] diff --git a/tutor/commands/upgrade/compose.py b/tutor/commands/upgrade/compose.py index ea0eaaf9dd4..7aef353bdaf 100644 --- a/tutor/commands/upgrade/compose.py +++ b/tutor/commands/upgrade/compose.py @@ -43,6 +43,9 @@ def upgrade_from(context: click.Context, from_release: str) -> None: upgrade_from_olive(context, config) running_release = "palm" + if running_release == "palm": + running_release = "quince" + def upgrade_from_ironwood(context: click.Context, config: Config) -> None: click.echo(fmt.title("Upgrading from Ironwood")) diff --git a/tutor/commands/upgrade/k8s.py b/tutor/commands/upgrade/k8s.py index b84ac4f300f..64cebaf354d 100644 --- a/tutor/commands/upgrade/k8s.py +++ b/tutor/commands/upgrade/k8s.py @@ -42,6 +42,9 @@ def upgrade_from(context: click.Context, from_release: str) -> None: upgrade_from_olive(context.obj, config) running_release = "palm" + if running_release == "palm": + running_release = "quince" + def upgrade_from_ironwood(config: Config) -> None: upgrade_mongodb(config, "3.4.24", "3.4") diff --git a/tutor/config.py b/tutor/config.py index 35ee80149d5..b5f48f30c35 100644 --- a/tutor/config.py +++ b/tutor/config.py @@ -153,18 +153,6 @@ def _load_config_defaults_yml( return items -@hooks.Filters.CONFIG_DEFAULTS.add() -def _set_openedx_common_version_in_nightly( - items: list[tuple[str, t.Any]] -) -> list[tuple[str, t.Any]]: - # REMOVE-AFTER-v16 move this callback to the dedicated openedx/plugin.py module - from tutor.__about__ import __version_suffix__ - - if __version_suffix__ == "nightly": - items.append(("OPENEDX_COMMON_VERSION", "master")) - return items - - def get_template(filename: str) -> Config: """ Get one of the configuration templates. diff --git a/tutor/env.py b/tutor/env.py index dd29b0c6038..cccce94d0d5 100644 --- a/tutor/env.py +++ b/tutor/env.py @@ -54,9 +54,6 @@ def _prepare_environment() -> None: ("HOST_USER_ID", utils.get_user_id()), ("TUTOR_APP", __app__.replace("-", "_")), ("TUTOR_VERSION", __version__), - # BuildKit used to be optional. Now, it's always enabled. - # This constant is just for temporary backwards compatibility (REMOVE-AFTER-V16). - ("is_buildkit_enabled", lambda: True), ("is_docker_rootless", utils.is_docker_rootless), ], ) @@ -460,6 +457,7 @@ def get_release(version: str) -> str: "14": "nutmeg", "15": "olive", "16": "palm", + "17": "quince", }[version.split(".", maxsplit=1)[0]] diff --git a/tutor/hooks/catalog.py b/tutor/hooks/catalog.py index e6dec94c522..56deed4e5ed 100644 --- a/tutor/hooks/catalog.py +++ b/tutor/hooks/catalog.py @@ -334,7 +334,6 @@ def your_filter_callback(some_data): #: - ``HOST_USER_ID``: the numerical ID of the user on the host. #: - ``TUTOR_APP``: the app name ("tutor" by default), used to determine the dev/local project names. #: - ``TUTOR_VERSION``: the current version of Tutor. - #: - ``is_buildkit_enabled``: a deprecated function which always returns ``True`` now. Will be removed after Quince. #: - ``iter_values_named``: a function to iterate on variables that start or end with a given string. #: - ``iter_mounts``: a function that yields compose-compatible bind-mounts for any given service. #: - ``iter_mounted_directories``: iterate on bind-mounted directory names. diff --git a/tutor/plugins/openedx.py b/tutor/plugins/openedx.py index 0bf2c041bc4..1ef4eff8221 100644 --- a/tutor/plugins/openedx.py +++ b/tutor/plugins/openedx.py @@ -6,6 +6,16 @@ from tutor import bindmount from tutor import hooks +from tutor.__about__ import __version_suffix__ + + +@hooks.Filters.CONFIG_DEFAULTS.add() +def _set_openedx_common_version_in_nightly( + items: list[tuple[str, t.Any]] +) -> list[tuple[str, t.Any]]: + if __version_suffix__ == "nightly": + items.append(("OPENEDX_COMMON_VERSION", "master")) + return items @hooks.Filters.APP_PUBLIC_HOSTS.add() diff --git a/tutor/templates/apps/openedx/settings/partials/common_all.py b/tutor/templates/apps/openedx/settings/partials/common_all.py index c9b747a89f1..c088b97b17e 100644 --- a/tutor/templates/apps/openedx/settings/partials/common_all.py +++ b/tutor/templates/apps/openedx/settings/partials/common_all.py @@ -149,9 +149,11 @@ # These warnings are visible in simple commands and init tasks import warnings -from django.utils.deprecation import RemovedInDjango40Warning, RemovedInDjango41Warning -warnings.filterwarnings("ignore", category=RemovedInDjango40Warning) -warnings.filterwarnings("ignore", category=RemovedInDjango41Warning) + +from django.utils.deprecation import RemovedInDjango50Warning, RemovedInDjango51Warning +warnings.filterwarnings("ignore", category=RemovedInDjango50Warning) +warnings.filterwarnings("ignore", category=RemovedInDjango51Warning) + warnings.filterwarnings("ignore", category=DeprecationWarning, module="wiki.plugins.links.wiki_plugin") warnings.filterwarnings("ignore", category=DeprecationWarning, module="boto.plugin") warnings.filterwarnings("ignore", category=DeprecationWarning, module="botocore.vendored.requests.packages.urllib3._collections") diff --git a/tutor/templates/build/openedx/Dockerfile b/tutor/templates/build/openedx/Dockerfile index a5b14523875..0f00f31e431 100644 --- a/tutor/templates/build/openedx/Dockerfile +++ b/tutor/templates/build/openedx/Dockerfile @@ -1,4 +1,4 @@ -# syntax=docker/dockerfile:1.4 +# syntax=docker/dockerfile:1 ###### Minimal image with base system requirements for most stages FROM docker.io/ubuntu:20.04 as minimal LABEL maintainer="Overhang.io " @@ -24,9 +24,9 @@ RUN --mount=type=cache,target=/var/cache/apt,sharing=locked \ # Install pyenv # https://www.python.org/downloads/ # https://github.com/pyenv/pyenv/releases -ARG PYTHON_VERSION=3.8.15 +ARG PYTHON_VERSION=3.8.18 ENV PYENV_ROOT /opt/pyenv -RUN git clone https://github.com/pyenv/pyenv $PYENV_ROOT --branch v2.3.17 --depth 1 +RUN git clone https://github.com/pyenv/pyenv $PYENV_ROOT --branch v2.3.29 --depth 1 # Install Python RUN $PYENV_ROOT/bin/pyenv install $PYTHON_VERSION @@ -69,13 +69,13 @@ FROM scratch as mnt-{{ name }} ###### Download extra locales to /openedx/locale/contrib/locale FROM minimal as locales -ARG OPENEDX_I18N_VERSION={{ OPENEDX_COMMON_VERSION }} -RUN cd /tmp \ - && curl -L -o openedx-i18n.tar.gz https://github.com/openedx/openedx-i18n/archive/$OPENEDX_I18N_VERSION.tar.gz \ - && tar xzf /tmp/openedx-i18n.tar.gz \ - && mkdir -p /openedx/locale/contrib \ - && mv openedx-i18n-*/edx-platform/locale /openedx/locale/contrib \ - && rm -rf openedx-i18n* +# TODO upgrade to quince +# ARG OPENEDX_I18N_VERSION=release/quince +ARG OPENEDX_I18N_VERSION=open-release/palm.4 +ADD --keep-git-dir https://github.com/openedx/openedx-i18n.git#$OPENEDX_I18N_VERSION /tmp/openedx-i18n +RUN mkdir --parents /openedx/locale && \ + mv /tmp/openedx-i18n/edx-platform/locale /openedx/locale/contrib && \ + rm -rf /tmp/openedx-i18n/ ###### Install python requirements in virtualenv FROM python as python-requirements @@ -94,10 +94,7 @@ RUN --mount=type=cache,target=/openedx/.cache/pip,sharing=shared \ # https://pypi.org/project/setuptools/ # https://pypi.org/project/pip/ # https://pypi.org/project/wheel/ - setuptools==67.6.1 pip==23.0.1. wheel==0.40.0 - -# Install missing py2neo package that was abruptly trimmed from pypi -RUN pip install https://github.com/overhangio/py2neo/releases/download/2021.2.3/py2neo-2021.2.3.tar.gz + setuptools==68.2.2 pip==23.2.1. wheel==0.41.2 # Install base requirements RUN --mount=type=bind,from=edx-platform,source=/requirements/edx/base.txt,target=/openedx/edx-platform/requirements/edx/base.txt \ @@ -108,18 +105,14 @@ RUN --mount=type=bind,from=edx-platform,source=/requirements/edx/base.txt,target RUN --mount=type=cache,target=/openedx/.cache/pip,sharing=shared \ pip install \ # Use redis as a django cache https://pypi.org/project/django-redis/ - django-redis==5.2.0 \ + django-redis==5.4.0 \ # uwsgi server https://pypi.org/project/uWSGI/ - uwsgi==2.0.21 + uwsgi==2.0.22 {{ patch("openedx-dockerfile-post-python-requirements") }} -# Install private requirements: this is useful for installing custom xblocks. -COPY ./requirements/ /openedx/requirements -RUN --mount=type=cache,target=/openedx/.cache/pip,sharing=shared \ - cd /openedx/requirements/ \ - && touch ./private.txt \ - && pip install -r ./private.txt +# Install scorm xblock +RUN pip install "openedx-scorm-xblock>=17.0.0,<18.0.0" {% for extra_requirements in OPENEDX_EXTRA_PIP_REQUIREMENTS %} RUN --mount=type=cache,target=/openedx/.cache/pip,sharing=shared \ @@ -167,7 +160,6 @@ COPY --chown=app:app --from=edx-platform / /openedx/edx-platform COPY --chown=app:app --from=locales /openedx/locale /openedx/locale COPY --chown=app:app --from=python /opt/pyenv /opt/pyenv COPY --chown=app:app --from=python-requirements /openedx/venv /openedx/venv -COPY --chown=app:app --from=python-requirements /openedx/requirements /openedx/requirements COPY --chown=app:app --from=python-requirements /mnt /mnt COPY --chown=app:app --from=nodejs-requirements /openedx/nodeenv /openedx/nodeenv COPY --chown=app:app --from=nodejs-requirements /openedx/edx-platform/node_modules /openedx/node_modules @@ -273,9 +265,9 @@ USER app RUN --mount=type=cache,target=/openedx/.cache/pip,sharing=shared \ pip install -r requirements/edx/development.txt # https://pypi.org/project/ipdb/ -# https://pypi.org/project/ipython +# https://pypi.org/project/ipython (8.12.x for Python 3.8) RUN --mount=type=cache,target=/openedx/.cache/pip,sharing=shared \ - pip install ipdb==0.13.13 ipython==8.12.0 + pip install ipdb==0.13.13 ipython==8.12.3 {# Re-install mounted requirements, otherwise they will be superseded by upstream reqs #} {% for name in iter_mounted_directories(MOUNTS, "openedx") %} diff --git a/tutor/templates/build/openedx/requirements/private-sample.txt b/tutor/templates/build/openedx/requirements/private-sample.txt deleted file mode 100644 index a0ea63a612e..00000000000 --- a/tutor/templates/build/openedx/requirements/private-sample.txt +++ /dev/null @@ -1,6 +0,0 @@ -# Add your additional requirements, such as xblocks, to this file. For -# requirements coming from private repositories, clone the repository to this -# folder and then add your requirement with the `-e` flag. Ex: -# -# git clone git@myserver:myprivaterepo.git -# echo "-e ./myprivaterepo/" >> private.txt diff --git a/tutor/templates/build/openedx/revisions.yml b/tutor/templates/build/openedx/revisions.yml index 34724a9707f..c01e271e536 100644 --- a/tutor/templates/build/openedx/revisions.yml +++ b/tutor/templates/build/openedx/revisions.yml @@ -1 +1 @@ -EDX_PLATFORM_REVISION: palm +EDX_PLATFORM_REVISION: quince diff --git a/tutor/templates/config/base.yml b/tutor/templates/config/base.yml index e34095a61bf..b1d6a14afa7 100644 --- a/tutor/templates/config/base.yml +++ b/tutor/templates/config/base.yml @@ -6,8 +6,9 @@ MYSQL_ROOT_PASSWORD: "{{ 8|random_string }}" OPENEDX_MYSQL_PASSWORD: "{{ 8|random_string }}" OPENEDX_SECRET_KEY: "{{ 24|random_string }}" PLUGINS: - # The MFE plugin is required + # enabled by default - mfe + - indigo PLUGIN_INDEXES: # Indexes in this list will be suffixed with the Open edX named version and # "plugins.yml". E.g: https://overhang.io/tutor/main/olive/plugins.yml diff --git a/tutor/templates/config/defaults.yml b/tutor/templates/config/defaults.yml index c75332812ad..1365722a7c5 100644 --- a/tutor/templates/config/defaults.yml +++ b/tutor/templates/config/defaults.yml @@ -12,12 +12,18 @@ DOCKER_COMPOSE_VERSION: "3.7" DOCKER_REGISTRY: "docker.io/" DOCKER_IMAGE_OPENEDX: "{{ DOCKER_REGISTRY }}overhangio/openedx:{{ TUTOR_VERSION }}" DOCKER_IMAGE_OPENEDX_DEV: "openedx-dev:{{ TUTOR_VERSION }}" -DOCKER_IMAGE_CADDY: "docker.io/caddy:2.6.4" -DOCKER_IMAGE_ELASTICSEARCH: "docker.io/elasticsearch:7.17.9" -DOCKER_IMAGE_MONGODB: "docker.io/mongo:4.4.22" +# https://hub.docker.com/_/caddy/tags +DOCKER_IMAGE_CADDY: "docker.io/caddy:2.7.4" +# https://hub.docker.com/_/elasticsearch/tags +DOCKER_IMAGE_ELASTICSEARCH: "docker.io/elasticsearch:7.17.13" +# https://hub.docker.com/_/mongo/tags +DOCKER_IMAGE_MONGODB: "docker.io/mongo:4.4.25" +# https://hub.docker.com/_/mysql/tags DOCKER_IMAGE_MYSQL: "docker.io/mysql:8.1.0" DOCKER_IMAGE_PERMISSIONS: "{{ DOCKER_REGISTRY }}overhangio/openedx-permissions:{{ TUTOR_VERSION }}" -DOCKER_IMAGE_REDIS: "docker.io/redis:7.0.11" +# https://hub.docker.com/_/redis/tags +DOCKER_IMAGE_REDIS: "docker.io/redis:7.2.1" +# https://hub.docker.com/r/devture/exim-relay/tags DOCKER_IMAGE_SMTP: "docker.io/devture/exim-relay:4.96-r1-0" EDX_PLATFORM_REPOSITORY: "https://github.com/openedx/edx-platform.git" EDX_PLATFORM_VERSION: "{{ OPENEDX_COMMON_VERSION }}" @@ -53,9 +59,8 @@ OPENEDX_LMS_UWSGI_WORKERS: 2 OPENEDX_MYSQL_DATABASE: "openedx" OPENEDX_MYSQL_USERNAME: "openedx" # the common version will be automatically set to "master" in the nightly branch -OPENEDX_COMMON_VERSION: "open-release/palm.4" -OPENEDX_EXTRA_PIP_REQUIREMENTS: - - "openedx-scorm-xblock>=16.0.0,<17.0.0" +OPENEDX_COMMON_VERSION: "open-release/quince.master" +OPENEDX_EXTRA_PIP_REQUIREMENTS: [] MYSQL_HOST: "mysql" MYSQL_PORT: 3306 MYSQL_ROOT_USERNAME: "root" diff --git a/tutor/templates/dev/docker-compose.jobs.yml b/tutor/templates/dev/docker-compose.jobs.yml index c9df4e2a12a..5e82a455aa6 100644 --- a/tutor/templates/dev/docker-compose.jobs.yml +++ b/tutor/templates/dev/docker-compose.jobs.yml @@ -16,8 +16,6 @@ x-openedx-job-service: - ../apps/openedx/config:/openedx/config:ro # theme files - ../build/openedx/themes:/openedx/themes - # editable requirements - - ../build/openedx/requirements:/openedx/requirements services: diff --git a/tutor/templates/dev/docker-compose.yml b/tutor/templates/dev/docker-compose.yml index 7c864358fc8..90fbe842bd4 100644 --- a/tutor/templates/dev/docker-compose.yml +++ b/tutor/templates/dev/docker-compose.yml @@ -12,8 +12,6 @@ x-openedx-service: - ../apps/openedx/config:/openedx/config:ro # theme files - ../build/openedx/themes:/openedx/themes - # editable requirements - - ../build/openedx/requirements:/openedx/requirements services: permissions: diff --git a/tutor/templates/jobs/init/cms.sh b/tutor/templates/jobs/init/cms.sh index 2f732da22f1..1d420a48ab0 100644 --- a/tutor/templates/jobs/init/cms.sh +++ b/tutor/templates/jobs/init/cms.sh @@ -11,4 +11,8 @@ if [ -d /openedx/data/uploads/ ]; then mv /openedx/data/uploads/* /openedx/media/ rm -rf /openedx/data/uploads/ fi -fi \ No newline at end of file +fi + +# Create waffle switches to enable some features, if they have not been explicitly defined before +# Copy-paste of units in Studio (highly requested new feature, but defaults to off in Quince) +(./manage.py cms waffle_flag --list | grep contentstore.enable_copy_paste_units) || ./manage.py lms waffle_flag --create contentstore.enable_copy_paste_units --everyone