diff --git a/CHANGELOG.md b/CHANGELOG.md index 90ab33e1bf2..ecebc14c4b0 100644 --- a/CHANGELOG.md +++ b/CHANGELOG.md @@ -20,6 +20,20 @@ instructions, because git commits are used to generate release notes: + +## v17.0.0 (2023-12-09) + +- 💥[Feature] Upgrade to Quince. (by @regisb) +- 💥[Feature] Replace "*.local.overhang.io" hostnames by "*.local.edly.io". (by @regisb) +- 💥[Feature] Enable the Indigo theme by default, if no other theme is set. (by @regisb) +- 💥[Deprecation] Tutor no longer supports the legacy Docker builder, which was previously available by setting ``DOCKER_BUILDKIT=0`` in the host environment. Going forward, Tutor will always use BuildKit (a.k.a. ``docker buildx`` in Docker v19-v22, or just ``docker build`` in Docker v23). This transition will improve build performance and should be seamless for Tutor users who are running a supported Docker version (by @kdmccormick). +- 💥[Deprecation] The template variable ``is_buildkit_enabled``, which now always returns True, is deprecated. Plugin authors should assume BuildKit is enabled and remove the variable from their templates (by @kdmccormick). +- [Bugfix] Updated how the Tutor setting `JWT_RSA_PRIVATE_KEY` is rendered into the LMS Django setting `JWT_AUTH['JWT_PRIVATE_SIGNING_JWK']` as required by a recent breaking upstream change. The new representation of the `JWT_PRIVATE_SIGNING_JWK` simply adds the `dq`, `dp`, and `qi` parameters. Without this fix, LMS would encounter an `InvalidKeyError` on all logins. (by @kdmccormick) +- [Improvement] You don't have to run `tutor config save` every time you enable or disable a plugin anymore. (by @CodeWithEmad) +- [Improvement] When configured with `RUN_MYSQL: true`, run `mysqld` with binlog expiry set to 3 days (rather than the default of 30). +- [Improvement] Fix `ulimits` error for elasticsearch in Docker rootless mode (by @OmarIthawi) +- [Improvement] Do not hardcode `OPENEDX_COMMON_VERSION = master` in the nightly branch. This removes git conflicts whenever we bump the common version in the master branch. (by @regisb) + ## v16.1.7 (2023-11-17) diff --git a/changelog.d/20230818_112124_kyle_buildkit.md b/changelog.d/20230818_112124_kyle_buildkit.md deleted file mode 100644 index 7084b290415..00000000000 --- a/changelog.d/20230818_112124_kyle_buildkit.md +++ /dev/null @@ -1,2 +0,0 @@ -- [Deprecation] The template variable ``is_buildkit_enabled``, which now always returns True, is deprecated. Plugin authors should assume BuildKit is enabled and remove the variable from their templates (by @kdmccormick). -- 💥[Deprecation] Tutor no longer supports the legacy Docker builder, which was previously available by setting ``DOCKER_BUILDKIT=0`` in the host environment. Going forward, Tutor will always use BuildKit (a.k.a. ``docker buildx`` in Docker v19-v22, or just ``docker build`` in Docker v23). This transition will improve build performance and should be seamless for Tutor users who are running a supported Docker version (by @kdmccormick). diff --git a/changelog.d/20231011_000434_codewithemad_save_configs.md b/changelog.d/20231011_000434_codewithemad_save_configs.md deleted file mode 100644 index 0afbad42ead..00000000000 --- a/changelog.d/20231011_000434_codewithemad_save_configs.md +++ /dev/null @@ -1 +0,0 @@ -- [Improvement] You don't have to run `tutor config save` every time you enable or disable a plugin anymore. (by @CodeWithEmad) \ No newline at end of file diff --git a/changelog.d/20231020_143112_kyle_jwk_full_complement.md b/changelog.d/20231020_143112_kyle_jwk_full_complement.md deleted file mode 100644 index 162b89db405..00000000000 --- a/changelog.d/20231020_143112_kyle_jwk_full_complement.md +++ /dev/null @@ -1 +0,0 @@ -- [Bugfix] Updated how the Tutor setting ``JWT_RSA_PRIVATE_KEY`` is rendered into the LMS Django setting ``JWT_AUTH['JWT_PRIVATE_SIGNING_JWK']`` as required by a recent breaking upstream change. The new representation of the ``JWT_PRIVATE_SIGNING_JWK`` simply adds the ``dq``, ``dp``, and ``qi`` parameters. Without this fix, LMS would encounter an ``InvalidKeyError`` on all logins (by @kdmccormick). diff --git a/changelog.d/20231023_155558_fghaas_binlog_expiry.md b/changelog.d/20231023_155558_fghaas_binlog_expiry.md deleted file mode 100644 index 95531a521c8..00000000000 --- a/changelog.d/20231023_155558_fghaas_binlog_expiry.md +++ /dev/null @@ -1 +0,0 @@ -[Improvement] When configured with `RUN_MYSQL: true`, run `mysqld` with binlog expiry set to 3 days (rather than the default of 30). diff --git a/changelog.d/20231116_081945_i_rootless_docker.md b/changelog.d/20231116_081945_i_rootless_docker.md deleted file mode 100644 index 717e3a24a36..00000000000 --- a/changelog.d/20231116_081945_i_rootless_docker.md +++ /dev/null @@ -1,2 +0,0 @@ -- [Improvement] Fix `ulimits` error for elasticsearch in Docker rootless mode (by @OmarIthawi) - diff --git a/changelog.d/20231120_160115_regis_nightly_common_version.md b/changelog.d/20231120_160115_regis_nightly_common_version.md deleted file mode 100644 index 4b5b0b3150b..00000000000 --- a/changelog.d/20231120_160115_regis_nightly_common_version.md +++ /dev/null @@ -1 +0,0 @@ -- [Improvement] Do not hardcode `OPENEDX_COMMON_VERSION = master` in the nightly branch. This removes git conflicts whenever we bump the common version in the master branch. (by @regisb) diff --git a/changelog.d/20231205_113430_regis_local_edly_io.md b/changelog.d/20231205_113430_regis_local_edly_io.md deleted file mode 100644 index dec279fb6b6..00000000000 --- a/changelog.d/20231205_113430_regis_local_edly_io.md +++ /dev/null @@ -1 +0,0 @@ -- 💥[Feature] Replace "*.local.overhang.io" hostnames by "*.local.edly.io". (by @regisb) diff --git a/docs/configuration.rst b/docs/configuration.rst index 2efd18054f8..94f54753a18 100644 --- a/docs/configuration.rst +++ b/docs/configuration.rst @@ -136,7 +136,7 @@ Open edX customisation This defines the git repository from which you install Open edX platform code. If you run an Open edX fork with custom patches, set this to your own git repository. You may also override this configuration parameter at build time, by providing a ``--build-arg`` option. -- ``OPENEDX_COMMON_VERSION`` (default: ``"open-release/palm.4"``, or ``master`` in :ref:`nightly `) +- ``OPENEDX_COMMON_VERSION`` (default: ``"open-release/quince.1"``, or ``master`` in :ref:`nightly `) This defines the default version that will be pulled from all Open edX git repositories. @@ -156,9 +156,9 @@ These two configuration parameters define which Redis database to use for Open e .. _openedx_extra_pip_requirements: -- ``OPENEDX_EXTRA_PIP_REQUIREMENTS`` (default: ``["openedx-scorm-xblock>=16.0.0,<17.0.0"]``) +- ``OPENEDX_EXTRA_PIP_REQUIREMENTS`` (default: ``[]``) -This defines extra pip packages that are going to be installed for Open edX. +Define extra pip packages that are going to be installed for edx-platform. - ``NPM_REGISTRY`` (default: ``"https://registry.npmjs.org/"``) @@ -401,16 +401,16 @@ Note that your edx-platform version must be a fork of the latest release **tag** If you don't create your fork from this tag, you *will* have important compatibility issues with other services. In particular: -- Do not try to run a fork from an older (pre-Palm) version of edx-platform: this will simply not work. +- Do not try to run a fork from an older (pre-Quince) version of edx-platform: this will simply not work. - Do not try to run a fork from the edx-platform master branch: there is a 99% probability that it will fail. -- Do not try to run a fork from the open-release/palm.master branch: Tutor will attempt to apply security and bug fix patches that might already be included in the open-release/palm.master but which were not yet applied to the latest release tag. Patch application will thus fail if you base your fork from the open-release/palm.master branch. +- Do not try to run a fork from the open-release/quince.master branch: Tutor will attempt to apply security and bug fix patches that might already be included in the open-release/quince.master but which were not yet applied to the latest release tag. Patch application will thus fail if you base your fork from the open-release/quince.master branch. .. _i18n: Adding custom translations ~~~~~~~~~~~~~~~~~~~~~~~~~~ -If you are not running Open edX in English (``LANGUAGE_CODE`` default: ``"en"``), chances are that some strings will not be properly translated. In most cases, this is because not enough contributors have helped translate Open edX into your language. It happens! With Tutor, available translated languages include those that come bundled with `edx-platform `__ as well as those from `openedx-i18n `__. +If you are not running Open edX in English (``LANGUAGE_CODE`` default: ``"en"``), chances are that some strings will not be properly translated. In most cases, this is because not enough contributors have helped translate Open edX into your language. It happens! With Tutor, available translated languages include those that come bundled with `edx-platform `__ as well as those from `openedx-i18n `__. Tutor offers a relatively simple mechanism to add custom translations to the openedx Docker image. You should create a folder that corresponds to your language code in the "build/openedx/locale" folder of the Tutor environment. This folder should contain a "LC_MESSAGES" folder. For instance:: @@ -431,9 +431,9 @@ Then, add a "django.po" file there that will contain your custom translations:: .. warning:: Don't forget to specify the file ``Content-Type`` when adding message strings with non-ASCII characters; otherwise a ``UnicodeDecodeError`` will be raised during compilation. -The "String to translate" part should match *exactly* the string that you would like to translate. You cannot make it up! The best way to find this string is to copy-paste it from the `upstream django.po file for the English language `__. +The "String to translate" part should match *exactly* the string that you would like to translate. You cannot make it up! The best way to find this string is to copy-paste it from the `upstream django.po file for the English language `__. -If you cannot find the string to translate in this file, then it means that you are trying to translate a string that is used in some piece of javascript code. Those strings are stored in a different file named "djangojs.po". You can check it out `in the edx-platform repo as well `__. Your custom javascript strings should also be stored in a "djangojs.po" file that should be placed in the same directory. +If you cannot find the string to translate in this file, then it means that you are trying to translate a string that is used in some piece of javascript code. Those strings are stored in a different file named "djangojs.po". You can check it out `in the edx-platform repo as well `__. Your custom javascript strings should also be stored in a "djangojs.po" file that should be placed in the same directory. To recap, here is an example. To translate a few strings in French, both from django.po and djangojs.po, we would have the following file hierarchy:: diff --git a/docs/dev.rst b/docs/dev.rst index d4224270a7d..2a1345edab0 100644 --- a/docs/dev.rst +++ b/docs/dev.rst @@ -126,7 +126,7 @@ The ``openedx-dev`` Docker image is based on the same ``openedx`` image used by - The user that runs inside the container has the same UID as the user on the host, to avoid permission problems inside mounted volumes (and in particular in the edx-platform repository). - Additional Python and system requirements are installed for convenient debugging: `ipython `__, `ipdb `__, vim, telnet. -- The edx-platform `development requirements `__ are installed. +- The edx-platform `development requirements `__ are installed. If you are using a custom ``openedx`` image, then you will need to rebuild ``openedx-dev`` every time you modify ``openedx``. To so, run:: diff --git a/docs/faq.rst b/docs/faq.rst index f4907d854c8..6467ecc4ac9 100644 --- a/docs/faq.rst +++ b/docs/faq.rst @@ -38,7 +38,7 @@ The `devstack `_ is meant for development o Is Tutor officially supported by edX? ------------------------------------- -Yes: as of the Open edX Maple release (December 9th 2021), Tutor is the only officially supported installation method for Open edX: see the `official installation instructions `__. +Yes: as of the Open edX Maple release (December 9th 2021), Tutor is the only officially supported installation method for Open edX: see the `official installation instructions `__. What features are missing from Tutor? ------------------------------------- diff --git a/docs/install.rst b/docs/install.rst index df4069261c2..b058ddced58 100644 --- a/docs/install.rst +++ b/docs/install.rst @@ -12,7 +12,7 @@ Requirements * Architecture: Both AMD64 and ARM64 are supported. * Required software: - - `Docker `__: v20.10.15+ + - `Docker `__: v24.0.5+ (with BuildKit 0.11+) - `Docker Compose `__: v2.0.0+ .. warning:: @@ -123,11 +123,11 @@ Major Open edX releases are published twice a year, in June and December, by the 4. Test the new release in a sandboxed environment. 5. If you are running edx-platform, or some other repository from a custom branch, then you should rebase (and test) your changes on top of the latest release tag (see :ref:`edx_platform_fork`). -The process for upgrading from one major release to the next works similarly to any other upgrade, with the ``launch`` command (see above). The single difference is that if the ``launch`` command detects that your tutor environment was generated with an older release, it will perform a few release-specific upgrade steps. These extra upgrade steps will be performed just once. But they will be ignored if you updated your local environment (for instance: with ``tutor config save``) before running ``launch``. This situation typically occurs if you need to re-build some Docker images (see above). In such a case, you should make use of the ``upgrade`` command. For instance, to upgrade a local installation from Olive to Palm and rebuild some Docker images, run:: +The process for upgrading from one major release to the next works similarly to any other upgrade, with the ``launch`` command (see above). The single difference is that if the ``launch`` command detects that your tutor environment was generated with an older release, it will perform a few release-specific upgrade steps. These extra upgrade steps will be performed just once. But they will be ignored if you updated your local environment (for instance: with ``tutor config save``) before running ``launch``. This situation typically occurs if you need to re-build some Docker images (see above). In such a case, you should make use of the ``upgrade`` command. For instance, to upgrade a local installation from Palm to Quince and rebuild some Docker images, run:: tutor config save tutor images build all # list the images that should be rebuilt here - tutor local upgrade --from=olive + tutor local upgrade --from=palm tutor local launch .. _autocomplete: diff --git a/docs/quickstart.rst b/docs/quickstart.rst index cad6dd20773..58cf4d4e32f 100644 --- a/docs/quickstart.rst +++ b/docs/quickstart.rst @@ -22,7 +22,7 @@ Yes :) This is what happens when you run ``tutor local launch``: 2. Configuration files are generated from templates. 3. Docker images are downloaded. 4. Docker containers are provisioned. -5. A full, production-ready Open edX platform (`Palm `__ release) is run with docker-compose. +5. A full, production-ready Open edX platform (`Quince `__ release) is run with docker-compose. The whole procedure should require less than 10 minutes, on a server with good bandwidth. Note that your host environment will not be affected in any way, since everything runs inside docker containers. Root access is not even necessary. diff --git a/docs/reference/indexes.rst b/docs/reference/indexes.rst index db06fcfadb6..e9c337ea337 100644 --- a/docs/reference/indexes.rst +++ b/docs/reference/indexes.rst @@ -7,10 +7,10 @@ Plugin indexes are a great way to have your plugins discovered by other users. P Index file paths ================ -A plugin index is a yaml-formatted file. It can be stored on the web or on your computer. In both cases, the index file location must end with "/plugins.yml". For instance, the following are valid index locations if you run the Open edX "Palm" release: +A plugin index is a yaml-formatted file. It can be stored on the web or on your computer. In both cases, the index file location must end with "/plugins.yml". For instance, the following are valid index locations if you run the Open edX "Quince" release: -- https://overhang.io/tutor/main/palm/plugins.yml -- ``/path/to/your/local/index/palm/plugins.yml`` +- https://overhang.io/tutor/main/quince/plugins.yml +- ``/path/to/your/local/index/quince/plugins.yml`` To add either indexes, run the ``tutor plugins index add`` command without the suffix. For instance:: @@ -106,9 +106,9 @@ Manage plugins in development Plugin developers and maintainers often want to install local versions of their plugins. They usually achieve this with ``pip install -e /path/to/tutor-plugin``. We can improve that workflow by creating an index for local plugins:: # Create the plugin index directory - mkdir -p ~/localindex/palm/ + mkdir -p ~/localindex/quince/ # Edit the index - vim ~/localindex/palm/plugins.yml + vim ~/localindex/quince/plugins.yml Add the following to the index:: diff --git a/requirements/base.txt b/requirements/base.txt index e79b5bdcc25..53ea28465eb 100644 --- a/requirements/base.txt +++ b/requirements/base.txt @@ -6,27 +6,27 @@ # appdirs==1.4.4 # via -r requirements/base.in -cachetools==5.3.1 +cachetools==5.3.2 # via google-auth certifi==2023.7.22 # via # kubernetes # requests -charset-normalizer==3.2.0 +charset-normalizer==3.3.1 # via requests click==8.1.7 # via -r requirements/base.in -google-auth==2.22.0 +google-auth==2.23.3 # via kubernetes idna==3.4 # via requests jinja2==3.1.2 # via -r requirements/base.in -kubernetes==27.2.0 +kubernetes==28.1.0 # via -r requirements/base.in markupsafe==2.1.3 # via jinja2 -mypy==1.5.1 +mypy==1.6.1 # via -r requirements/base.in mypy-extensions==1.0.0 # via mypy @@ -40,7 +40,7 @@ pyasn1==0.5.0 # rsa pyasn1-modules==0.3.0 # via google-auth -pycryptodome==3.18.0 +pycryptodome==3.19.0 # via -r requirements/base.in python-dateutil==2.8.2 # via kubernetes @@ -58,19 +58,17 @@ rsa==4.9 # via google-auth six==1.16.0 # via - # google-auth # kubernetes # python-dateutil tomli==2.0.1 # via mypy -typing-extensions==4.7.1 +typing-extensions==4.8.0 # via # -r requirements/base.in # mypy urllib3==1.26.18 # via - # google-auth # kubernetes # requests -websocket-client==1.6.2 +websocket-client==1.6.4 # via kubernetes diff --git a/requirements/dev.in b/requirements/dev.in index c53771c4193..ef0d7ada7b6 100644 --- a/requirements/dev.in +++ b/requirements/dev.in @@ -7,6 +7,12 @@ pyinstaller scriv twine +# Constraints +# for compatibility with sphinx-rtd-theme +# drop this constraint once sphinx-rtd-theme 2.0.0 comes out +# https://github.com/readthedocs/sphinx_rtd_theme/tags +docutils<0.19.0 + # Types packages types-docutils types-PyYAML diff --git a/requirements/dev.txt b/requirements/dev.txt index 8d8b265839a..fd7ba3e7479 100644 --- a/requirements/dev.txt +++ b/requirements/dev.txt @@ -4,21 +4,19 @@ # # pip-compile requirements/dev.in # -altgraph==0.17.3 +altgraph==0.17.4 # via pyinstaller appdirs==1.4.4 # via -r requirements/base.txt -astroid==2.15.6 +astroid==3.0.1 # via pylint attrs==23.1.0 # via scriv -black==23.7.0 +black==23.10.1 # via -r requirements/dev.in -bleach==6.0.0 - # via readme-renderer -build==0.10.0 +build==1.0.3 # via pip-tools -cachetools==5.3.1 +cachetools==5.3.2 # via # -r requirements/base.txt # google-auth @@ -27,9 +25,9 @@ certifi==2023.7.22 # -r requirements/base.txt # kubernetes # requests -cffi==1.15.1 +cffi==1.16.0 # via cryptography -charset-normalizer==3.2.0 +charset-normalizer==3.3.1 # via # -r requirements/base.txt # requests @@ -42,15 +40,17 @@ click==8.1.7 # scriv click-log==0.4.0 # via scriv -coverage==7.3.0 +coverage==7.3.2 # via -r requirements/dev.in cryptography==41.0.7 # via secretstorage dill==0.3.7 # via pylint -docutils==0.20.1 - # via readme-renderer -google-auth==2.22.0 +docutils==0.18.1 + # via + # -r requirements/dev.in + # readme-renderer +google-auth==2.23.3 # via # -r requirements/base.txt # kubernetes @@ -60,7 +60,9 @@ idna==3.4 # requests importlib-metadata==6.8.0 # via + # build # keyring + # pyinstaller # twine importlib-resources==6.1.1 # via keyring @@ -78,12 +80,12 @@ jinja2==3.1.2 # scriv keyring==24.2.0 # via twine -kubernetes==27.2.0 +kubernetes==28.1.0 # via -r requirements/base.txt -lazy-object-proxy==1.9.0 - # via astroid markdown-it-py==3.0.0 - # via rich + # via + # rich + # scriv markupsafe==2.1.3 # via # -r requirements/base.txt @@ -94,29 +96,32 @@ mdurl==0.1.2 # via markdown-it-py more-itertools==10.1.0 # via jaraco-classes -mypy==1.5.1 +mypy==1.6.1 # via -r requirements/base.txt mypy-extensions==1.0.0 # via # -r requirements/base.txt # black # mypy +nh3==0.2.14 + # via readme-renderer oauthlib==3.2.2 # via # -r requirements/base.txt # kubernetes # requests-oauthlib -packaging==23.1 +packaging==23.2 # via # black # build + # pyinstaller pathspec==0.11.2 # via black pip-tools==7.3.0 # via -r requirements/dev.in pkginfo==1.9.6 # via twine -platformdirs==3.10.0 +platformdirs==3.11.0 # via # black # pylint @@ -131,17 +136,17 @@ pyasn1-modules==0.3.0 # google-auth pycparser==2.21 # via cffi -pycryptodome==3.18.0 +pycryptodome==3.19.0 # via -r requirements/base.txt pygments==2.16.1 # via # readme-renderer # rich -pyinstaller==5.13.1 +pyinstaller==6.1.0 # via -r requirements/dev.in -pyinstaller-hooks-contrib==2023.7 +pyinstaller-hooks-contrib==2023.10 # via pyinstaller -pylint==2.17.5 +pylint==3.0.2 # via -r requirements/dev.in pyproject-hooks==1.0.0 # via build @@ -153,7 +158,7 @@ pyyaml==6.0.1 # via # -r requirements/base.txt # kubernetes -readme-renderer==41.0 +readme-renderer==42.0 # via twine requests==2.31.0 # via @@ -171,21 +176,19 @@ requests-toolbelt==1.0.0 # via twine rfc3986==2.0.0 # via twine -rich==13.5.2 +rich==13.6.0 # via twine rsa==4.9 # via # -r requirements/base.txt # google-auth -scriv==1.3.1 +scriv==1.5.0 # via -r requirements/dev.in secretstorage==3.3.3 # via keyring six==1.16.0 # via # -r requirements/base.txt - # bleach - # google-auth # kubernetes # python-dateutil tomli==2.0.1 @@ -203,11 +206,11 @@ twine==4.0.2 # via -r requirements/dev.in types-docutils==0.20.0.3 # via -r requirements/dev.in -types-pyyaml==6.0.12.11 +types-pyyaml==6.0.12.12 # via -r requirements/dev.in -types-setuptools==68.1.0.0 +types-setuptools==68.2.0.0 # via -r requirements/dev.in -typing-extensions==4.7.1 +typing-extensions==4.8.0 # via # -r requirements/base.txt # astroid @@ -218,21 +221,16 @@ typing-extensions==4.7.1 urllib3==1.26.18 # via # -r requirements/base.txt - # google-auth # kubernetes # requests # twine -webencodings==0.5.1 - # via bleach -websocket-client==1.6.2 +websocket-client==1.6.4 # via # -r requirements/base.txt # kubernetes wheel==0.41.2 # via pip-tools -wrapt==1.15.0 - # via astroid -zipp==3.16.2 +zipp==3.17.0 # via # importlib-metadata # importlib-resources diff --git a/requirements/docs.txt b/requirements/docs.txt index 80bb0301101..c4f5c6e9c2d 100644 --- a/requirements/docs.txt +++ b/requirements/docs.txt @@ -8,9 +8,9 @@ alabaster==0.7.13 # via sphinx appdirs==1.4.4 # via -r requirements/base.txt -babel==2.12.1 +babel==2.13.1 # via sphinx -cachetools==5.3.1 +cachetools==5.3.2 # via # -r requirements/base.txt # google-auth @@ -19,7 +19,7 @@ certifi==2023.7.22 # -r requirements/base.txt # kubernetes # requests -charset-normalizer==3.2.0 +charset-normalizer==3.3.1 # via # -r requirements/base.txt # requests @@ -32,7 +32,7 @@ docutils==0.18.1 # sphinx # sphinx-click # sphinx-rtd-theme -google-auth==2.22.0 +google-auth==2.23.3 # via # -r requirements/base.txt # kubernetes @@ -48,13 +48,13 @@ jinja2==3.1.2 # via # -r requirements/base.txt # sphinx -kubernetes==27.2.0 +kubernetes==28.1.0 # via -r requirements/base.txt markupsafe==2.1.3 # via # -r requirements/base.txt # jinja2 -mypy==1.5.1 +mypy==1.6.1 # via -r requirements/base.txt mypy-extensions==1.0.0 # via @@ -65,7 +65,7 @@ oauthlib==3.2.2 # -r requirements/base.txt # kubernetes # requests-oauthlib -packaging==23.1 +packaging==23.2 # via sphinx pyasn1==0.5.0 # via @@ -76,7 +76,7 @@ pyasn1-modules==0.3.0 # via # -r requirements/base.txt # google-auth -pycryptodome==3.18.0 +pycryptodome==3.19.0 # via -r requirements/base.txt pygments==2.16.1 # via sphinx @@ -107,7 +107,6 @@ rsa==4.9 six==1.16.0 # via # -r requirements/base.txt - # google-auth # kubernetes # python-dateutil snowballstemmer==2.2.0 @@ -140,17 +139,16 @@ tomli==2.0.1 # via # -r requirements/base.txt # mypy -typing-extensions==4.7.1 +typing-extensions==4.8.0 # via # -r requirements/base.txt # mypy urllib3==1.26.18 # via # -r requirements/base.txt - # google-auth # kubernetes # requests -websocket-client==1.6.2 +websocket-client==1.6.4 # via # -r requirements/base.txt # kubernetes diff --git a/requirements/plugins.txt b/requirements/plugins.txt index e67affd113c..c78ef5773fb 100644 --- a/requirements/plugins.txt +++ b/requirements/plugins.txt @@ -1,12 +1,13 @@ -# change version ranges when upgrading from palm -tutor-android>=16.0.0,<17.0.0 -tutor-cairn>=16.0.0,<17.0.0 -tutor-discovery>=16.0.0,<17.0.0 -tutor-ecommerce>=16.0.0,<17.0.0 -tutor-forum>=16.0.0,<17.0.0 -tutor-license>=16.0.0,<17.0.0 -tutor-mfe>=16.0.0,<17.0.0 -tutor-minio>=16.0.0,<17.0.0 -tutor-notes>=16.0.0,<17.0.0 -tutor-webui>=16.0.0,<17.0.0 -tutor-xqueue>=16.0.0,<17.0.0 +# change version ranges when upgrading from quince +tutor-android>=17.0.0,<18.0.0 +tutor-cairn>=17.0.0,<18.0.0 +tutor-credentials>=17.0.0,<18.0.0 +tutor-discovery>=17.0.0,<18.0.0 +tutor-ecommerce>=17.0.0,<18.0.0 +tutor-forum>=17.0.0,<18.0.0 +tutor-jupyter>=17.0.0,<18.0.0 +tutor-mfe>=17.0.0,<18.0.0 +tutor-minio>=17.0.0,<18.0.0 +tutor-notes>=17.0.0,<18.0.0 +tutor-webui>=17.0.0,<18.0.0 +tutor-xqueue>=17.0.0,<18.0.0 diff --git a/tests/commands/test_config.py b/tests/commands/test_config.py index 5c9dcea307a..58b15edd93a 100644 --- a/tests/commands/test_config.py +++ b/tests/commands/test_config.py @@ -94,9 +94,8 @@ def test_config_append_with_existing_default(self) -> None: ) config = tutor_config.load(root) assert isinstance(config["OPENEDX_EXTRA_PIP_REQUIREMENTS"], list) - self.assertEqual(2, len(config["OPENEDX_EXTRA_PIP_REQUIREMENTS"])) self.assertEqual( - "my-package==1.0.0", config["OPENEDX_EXTRA_PIP_REQUIREMENTS"][1] + ["my-package==1.0.0"], config["OPENEDX_EXTRA_PIP_REQUIREMENTS"] ) diff --git a/tests/test_env.py b/tests/test_env.py index a79060991a3..27cff9678bd 100644 --- a/tests/test_env.py +++ b/tests/test_env.py @@ -260,7 +260,7 @@ def test_current_version_in_latest_env(self) -> None: ) as f: f.write(__version__) self.assertEqual(__version__, env.current_version(root)) - self.assertEqual("palm", env.get_env_release(root)) + self.assertEqual("quince", env.get_env_release(root)) self.assertIsNone(env.should_upgrade_from_release(root)) self.assertTrue(env.is_up_to_date(root)) diff --git a/tutor/__about__.py b/tutor/__about__.py index 3283265a18c..33297dd3a4d 100644 --- a/tutor/__about__.py +++ b/tutor/__about__.py @@ -2,7 +2,7 @@ # Increment this version number to trigger a new release. See # docs/tutor.html#versioning for information on the versioning scheme. -__version__ = "16.1.7" +__version__ = "17.0.0" # The version suffix will be appended to the actual version, separated by a # dash. Use this suffix to differentiate between the actual released version and @@ -10,7 +10,7 @@ # the nightly branch. # The suffix is cleanly separated from the __version__ in this module to avoid # conflicts when merging branches. -__version_suffix__ = "nightly" +__version_suffix__ = "" # The app name will be used to define the name of the default tutor root and # plugin directory. To avoid conflicts between multiple locally-installed diff --git a/tutor/commands/upgrade/__init__.py b/tutor/commands/upgrade/__init__.py index 9c1d2034249..b7a4e028dec 100644 --- a/tutor/commands/upgrade/__init__.py +++ b/tutor/commands/upgrade/__init__.py @@ -1,4 +1,4 @@ -# Note: don't forget to change this when we upgrade from palm +# Note: don't forget to change this when we upgrade from quince OPENEDX_RELEASE_NAMES = [ "ironwood", "juniper", @@ -7,4 +7,6 @@ "maple", "nutmeg", "olive", + "palm", + "quince", ] diff --git a/tutor/commands/upgrade/compose.py b/tutor/commands/upgrade/compose.py index ea0eaaf9dd4..7aef353bdaf 100644 --- a/tutor/commands/upgrade/compose.py +++ b/tutor/commands/upgrade/compose.py @@ -43,6 +43,9 @@ def upgrade_from(context: click.Context, from_release: str) -> None: upgrade_from_olive(context, config) running_release = "palm" + if running_release == "palm": + running_release = "quince" + def upgrade_from_ironwood(context: click.Context, config: Config) -> None: click.echo(fmt.title("Upgrading from Ironwood")) diff --git a/tutor/commands/upgrade/k8s.py b/tutor/commands/upgrade/k8s.py index b84ac4f300f..64cebaf354d 100644 --- a/tutor/commands/upgrade/k8s.py +++ b/tutor/commands/upgrade/k8s.py @@ -42,6 +42,9 @@ def upgrade_from(context: click.Context, from_release: str) -> None: upgrade_from_olive(context.obj, config) running_release = "palm" + if running_release == "palm": + running_release = "quince" + def upgrade_from_ironwood(config: Config) -> None: upgrade_mongodb(config, "3.4.24", "3.4") diff --git a/tutor/env.py b/tutor/env.py index dd29b0c6038..6602d8d3538 100644 --- a/tutor/env.py +++ b/tutor/env.py @@ -460,6 +460,7 @@ def get_release(version: str) -> str: "14": "nutmeg", "15": "olive", "16": "palm", + "17": "quince", }[version.split(".", maxsplit=1)[0]] diff --git a/tutor/templates/apps/openedx/settings/partials/common_all.py b/tutor/templates/apps/openedx/settings/partials/common_all.py index c9b747a89f1..749f33420ef 100644 --- a/tutor/templates/apps/openedx/settings/partials/common_all.py +++ b/tutor/templates/apps/openedx/settings/partials/common_all.py @@ -149,9 +149,16 @@ # These warnings are visible in simple commands and init tasks import warnings -from django.utils.deprecation import RemovedInDjango40Warning, RemovedInDjango41Warning -warnings.filterwarnings("ignore", category=RemovedInDjango40Warning) -warnings.filterwarnings("ignore", category=RemovedInDjango41Warning) + +try: + from django.utils.deprecation import RemovedInDjango50Warning, RemovedInDjango51Warning + warnings.filterwarnings("ignore", category=RemovedInDjango50Warning) + warnings.filterwarnings("ignore", category=RemovedInDjango51Warning) +except ImportError: + from django.utils.deprecation import RemovedInDjango40Warning, RemovedInDjango41Warning + warnings.filterwarnings("ignore", category=RemovedInDjango40Warning) + warnings.filterwarnings("ignore", category=RemovedInDjango41Warning) + warnings.filterwarnings("ignore", category=DeprecationWarning, module="wiki.plugins.links.wiki_plugin") warnings.filterwarnings("ignore", category=DeprecationWarning, module="boto.plugin") warnings.filterwarnings("ignore", category=DeprecationWarning, module="botocore.vendored.requests.packages.urllib3._collections") diff --git a/tutor/templates/build/openedx/Dockerfile b/tutor/templates/build/openedx/Dockerfile index 0f0aa66f4ed..a1a3894dd8b 100644 --- a/tutor/templates/build/openedx/Dockerfile +++ b/tutor/templates/build/openedx/Dockerfile @@ -1,4 +1,4 @@ -# syntax=docker/dockerfile:1.4 +# syntax=docker/dockerfile:1 ###### Minimal image with base system requirements for most stages FROM docker.io/ubuntu:20.04 as minimal LABEL maintainer="Overhang.io " @@ -24,9 +24,9 @@ RUN --mount=type=cache,target=/var/cache/apt,sharing=locked \ # Install pyenv # https://www.python.org/downloads/ # https://github.com/pyenv/pyenv/releases -ARG PYTHON_VERSION=3.8.15 +ARG PYTHON_VERSION=3.8.18 ENV PYENV_ROOT /opt/pyenv -RUN git clone https://github.com/pyenv/pyenv $PYENV_ROOT --branch v2.3.17 --depth 1 +RUN git clone https://github.com/pyenv/pyenv $PYENV_ROOT --branch v2.3.29 --depth 1 # Install Python RUN $PYENV_ROOT/bin/pyenv install $PYTHON_VERSION @@ -64,13 +64,13 @@ COPY --from=code /openedx/edx-platform / ###### Download extra locales to /openedx/locale/contrib/locale FROM minimal as locales -ARG OPENEDX_I18N_VERSION={{ OPENEDX_COMMON_VERSION }} -RUN cd /tmp \ - && curl -L -o openedx-i18n.tar.gz https://github.com/openedx/openedx-i18n/archive/$OPENEDX_I18N_VERSION.tar.gz \ - && tar xzf /tmp/openedx-i18n.tar.gz \ - && mkdir -p /openedx/locale/contrib \ - && mv openedx-i18n-*/edx-platform/locale /openedx/locale/contrib \ - && rm -rf openedx-i18n* +# TODO upgrade to quince +# ARG OPENEDX_I18N_VERSION=release/quince +ARG OPENEDX_I18N_VERSION=open-release/palm.4 +ADD --keep-git-dir https://github.com/openedx/openedx-i18n.git#$OPENEDX_I18N_VERSION /tmp/openedx-i18n +RUN mkdir --parents /openedx/locale && \ + mv /tmp/openedx-i18n/edx-platform/locale /openedx/locale/contrib && \ + rm -rf /tmp/openedx-i18n/ ###### Install python requirements in virtualenv FROM python as python-requirements @@ -89,10 +89,7 @@ RUN --mount=type=cache,target=/openedx/.cache/pip,sharing=shared \ # https://pypi.org/project/setuptools/ # https://pypi.org/project/pip/ # https://pypi.org/project/wheel/ - setuptools==67.6.1 pip==23.0.1. wheel==0.40.0 - -# Install missing py2neo package that was abruptly trimmed from pypi -RUN pip install https://github.com/overhangio/py2neo/releases/download/2021.2.3/py2neo-2021.2.3.tar.gz + setuptools==68.2.2 pip==23.2.1. wheel==0.41.2 # Install base requirements RUN --mount=type=bind,from=edx-platform,source=/requirements/edx/base.txt,target=/openedx/edx-platform/requirements/edx/base.txt \ @@ -103,12 +100,15 @@ RUN --mount=type=bind,from=edx-platform,source=/requirements/edx/base.txt,target RUN --mount=type=cache,target=/openedx/.cache/pip,sharing=shared \ pip install \ # Use redis as a django cache https://pypi.org/project/django-redis/ - django-redis==5.2.0 \ + django-redis==5.4.0 \ # uwsgi server https://pypi.org/project/uWSGI/ - uwsgi==2.0.21 + uwsgi==2.0.22 {{ patch("openedx-dockerfile-post-python-requirements") }} +# Install scorm xblock +RUN pip install "openedx-scorm-xblock>=17.0.0,<18.0.0" + # Install private requirements: this is useful for installing custom xblocks. COPY ./requirements/ /openedx/requirements RUN --mount=type=cache,target=/openedx/.cache/pip,sharing=shared \ @@ -261,9 +261,9 @@ USER app RUN --mount=type=cache,target=/openedx/.cache/pip,sharing=shared \ pip install -r requirements/edx/development.txt # https://pypi.org/project/ipdb/ -# https://pypi.org/project/ipython +# https://pypi.org/project/ipython (8.12.x for Python 3.8) RUN --mount=type=cache,target=/openedx/.cache/pip,sharing=shared \ - pip install ipdb==0.13.13 ipython==8.12.0 + pip install ipdb==0.13.13 ipython==8.12.3 # Add ipdb as default PYTHONBREAKPOINT ENV PYTHONBREAKPOINT=ipdb.set_trace diff --git a/tutor/templates/build/openedx/revisions.yml b/tutor/templates/build/openedx/revisions.yml index 34724a9707f..c01e271e536 100644 --- a/tutor/templates/build/openedx/revisions.yml +++ b/tutor/templates/build/openedx/revisions.yml @@ -1 +1 @@ -EDX_PLATFORM_REVISION: palm +EDX_PLATFORM_REVISION: quince diff --git a/tutor/templates/config/base.yml b/tutor/templates/config/base.yml index e34095a61bf..b1d6a14afa7 100644 --- a/tutor/templates/config/base.yml +++ b/tutor/templates/config/base.yml @@ -6,8 +6,9 @@ MYSQL_ROOT_PASSWORD: "{{ 8|random_string }}" OPENEDX_MYSQL_PASSWORD: "{{ 8|random_string }}" OPENEDX_SECRET_KEY: "{{ 24|random_string }}" PLUGINS: - # The MFE plugin is required + # enabled by default - mfe + - indigo PLUGIN_INDEXES: # Indexes in this list will be suffixed with the Open edX named version and # "plugins.yml". E.g: https://overhang.io/tutor/main/olive/plugins.yml diff --git a/tutor/templates/config/defaults.yml b/tutor/templates/config/defaults.yml index c75332812ad..1365722a7c5 100644 --- a/tutor/templates/config/defaults.yml +++ b/tutor/templates/config/defaults.yml @@ -12,12 +12,18 @@ DOCKER_COMPOSE_VERSION: "3.7" DOCKER_REGISTRY: "docker.io/" DOCKER_IMAGE_OPENEDX: "{{ DOCKER_REGISTRY }}overhangio/openedx:{{ TUTOR_VERSION }}" DOCKER_IMAGE_OPENEDX_DEV: "openedx-dev:{{ TUTOR_VERSION }}" -DOCKER_IMAGE_CADDY: "docker.io/caddy:2.6.4" -DOCKER_IMAGE_ELASTICSEARCH: "docker.io/elasticsearch:7.17.9" -DOCKER_IMAGE_MONGODB: "docker.io/mongo:4.4.22" +# https://hub.docker.com/_/caddy/tags +DOCKER_IMAGE_CADDY: "docker.io/caddy:2.7.4" +# https://hub.docker.com/_/elasticsearch/tags +DOCKER_IMAGE_ELASTICSEARCH: "docker.io/elasticsearch:7.17.13" +# https://hub.docker.com/_/mongo/tags +DOCKER_IMAGE_MONGODB: "docker.io/mongo:4.4.25" +# https://hub.docker.com/_/mysql/tags DOCKER_IMAGE_MYSQL: "docker.io/mysql:8.1.0" DOCKER_IMAGE_PERMISSIONS: "{{ DOCKER_REGISTRY }}overhangio/openedx-permissions:{{ TUTOR_VERSION }}" -DOCKER_IMAGE_REDIS: "docker.io/redis:7.0.11" +# https://hub.docker.com/_/redis/tags +DOCKER_IMAGE_REDIS: "docker.io/redis:7.2.1" +# https://hub.docker.com/r/devture/exim-relay/tags DOCKER_IMAGE_SMTP: "docker.io/devture/exim-relay:4.96-r1-0" EDX_PLATFORM_REPOSITORY: "https://github.com/openedx/edx-platform.git" EDX_PLATFORM_VERSION: "{{ OPENEDX_COMMON_VERSION }}" @@ -53,9 +59,8 @@ OPENEDX_LMS_UWSGI_WORKERS: 2 OPENEDX_MYSQL_DATABASE: "openedx" OPENEDX_MYSQL_USERNAME: "openedx" # the common version will be automatically set to "master" in the nightly branch -OPENEDX_COMMON_VERSION: "open-release/palm.4" -OPENEDX_EXTRA_PIP_REQUIREMENTS: - - "openedx-scorm-xblock>=16.0.0,<17.0.0" +OPENEDX_COMMON_VERSION: "open-release/quince.master" +OPENEDX_EXTRA_PIP_REQUIREMENTS: [] MYSQL_HOST: "mysql" MYSQL_PORT: 3306 MYSQL_ROOT_USERNAME: "root" diff --git a/tutor/templates/jobs/init/cms.sh b/tutor/templates/jobs/init/cms.sh index 2f732da22f1..1d420a48ab0 100644 --- a/tutor/templates/jobs/init/cms.sh +++ b/tutor/templates/jobs/init/cms.sh @@ -11,4 +11,8 @@ if [ -d /openedx/data/uploads/ ]; then mv /openedx/data/uploads/* /openedx/media/ rm -rf /openedx/data/uploads/ fi -fi \ No newline at end of file +fi + +# Create waffle switches to enable some features, if they have not been explicitly defined before +# Copy-paste of units in Studio (highly requested new feature, but defaults to off in Quince) +(./manage.py cms waffle_flag --list | grep contentstore.enable_copy_paste_units) || ./manage.py lms waffle_flag --create contentstore.enable_copy_paste_units --everyone