Initialize codejail_service application #2

timmc-edx · 2025-01-23T23:30:44Z

Two commits, best reviewed individually:

Straight-up output from a cookiecutter run
A fairly large commit that mostly deletes unwanted things and updates dependencies (but see its full description for many details)

This is part of edx/edx-arch-experiments#894

- Use cookiecutter version of .gitignore - Don't include LICENSE.txt (already have LICENSE)

Major changes: - Trim dependencies, settings, and application code to remove anything relating to having a database (auth, models, pii-annotations, reserved-keywords) or being exposed to the outside world (cors, csrf). Add a blank DB for tests. - Remove unneeded requirements files. We're going to deploy the base layer instead of the production layer, and requirements.txt is just not needed at all as far as I can tell. - Install gunicorn in the base layer. It's not clear if we'll want to use it from devstack (or just use runserver) but it should be available when we want to use it. - Don't hardcode gunicorn's bind address; we'll pass it as `--bind` from the Dockerfile. - Remove Dockerfile and devstack.py as these will be provided elsewhere. Fixes: - Use Python 3.12 for tox invocation (to match tox.ini and others) Docs: - Update catalog-info.yaml from defaults - Tweaks to PR template TODO: - Re-enable various edx-django-utils middleware as soon as I can determine how to get them working without a DB. (Some of them use waffle flags.) Finally: - Run `make upgrade`

robrap

Thanks for the work on this one.

robrap · 2025-01-24T21:49:51Z

Makefile

-	python manage.py check_reserved_keywords --override_file db_keyword_overrides.yml
-
-validate: test quality pii_check check_keywords ## run tests, quality, and PII annotation checks
+validate: test quality ## run tests, quality

 migrate: ## apply database migrations


Can this go away? Do we need a follow-up ticket on further db clean-up that we could point to in comments?

If so, do we want to search for all locations, or just leave the searching to the follow-up ticket?

robrap · 2025-01-24T21:51:29Z

Makefile

-
-pkg-devstack: ## build the codejail-service image from the latest configuration and code
-	docker build -t codejail-service:latest -f docker/build/codejail-service/Dockerfile git://github.com/openedx/configuration
-
 detect_changed_source_translations: ## check if translation files are up-to-date


Do we know how translations work in codejail, and whether this is needed? Should we add a low-priority follow-up issue in the repo?

robrap · 2025-01-24T21:53:23Z

Makefile

 detect_changed_source_translations: ## check if translation files are up-to-date
 	cd codejail_service && i18n_tool changed

 validate_translations: fake_translations detect_changed_source_translations ## install fake translations and check if translation files are up-to-date

-docker_build:


As noted elsewhere (in Slack), I'm unsure what kind of basic in-repo Docker support is still wanted at this point. I think it is fine to delete this while we don't know, and just pointing out that I don't know.

robrap · 2025-01-24T21:58:39Z

codejail_service/docker_gunicorn_configuration.py

@@ -5,7 +5,6 @@

 preload_app = True
 timeout = 300
-bind = "0.0.0.0:18030"


Should we disable codecov for this file?

robrap · 2025-01-24T22:07:24Z

codejail_service/settings/base.py

+    # TODO: re-enable once we have waffle available
+    # 'edx_django_utils.monitoring.DeploymentMonitoringMiddleware',  # python and django version
+    # 'edx_django_utils.monitoring.CookieMonitoringMiddleware',  # cookie names (compliance) and sizes
+    # 'edx_django_utils.monitoring.CachedCustomMonitoringMiddleware',  # support accumulate & increment
+    # 'edx_django_utils.monitoring.MonitoringMemoryMiddleware',  # memory usage


Changes I made below:

I updated the TODO. I don't see how the original made sense, but I might be missing something.

I removed CookieMonitoringMiddleware.

I added proposed priorities for re-enabling into the comments.

# TODO: re-enable once waffle. which depends on a db, is optional. <INSERT TICKET LINK> # 'edx_django_utils.monitoring.DeploymentMonitoringMiddleware', # python and django version (Re-enable Priority: P3) # 'edx_django_utils.monitoring.CachedCustomMonitoringMiddleware', # support accumulate & increment (Re-enable Priority: P5) # 'edx_django_utils.monitoring.MonitoringMemoryMiddleware', # memory usage (Re-enable Priority: P5)

robrap · 2025-01-24T22:13:36Z

codejail_service/settings/test.py

@@ -2,15 +2,10 @@

 from codejail_service.settings.base import *  # pylint: disable=wildcard-import

-# IN-MEMORY TEST DATABASE
+# Django *really* wants a database. So we give it a blank in-memory one.


Dang. Really? I wonder if we could rig it so it blows up if any unit test actually tries to use it?

robrap · 2025-01-24T22:14:12Z

codejail_service/urls.py

    re_path(r'^api/', include(api_urls)),
-    re_path(r'^api-docs/', get_swagger_view(title='codejail-service API')),


Could this be useful in a development environment?

robrap · 2025-01-24T22:18:17Z

requirements/base.in

@@ -2,17 +2,9 @@
 -c constraints.txt

 Django         # Web application framework
-django-cors-headers
-django-extensions
-django-rest-swagger


Again - a question mark.

robrap · 2025-01-24T22:19:18Z

requirements/base.in

-django-extensions
-django-rest-swagger
-django-waffle
-djangorestframework


I don't know if DRF will still make sense for consistency or not.

robrap · 2025-01-24T22:19:39Z

requirements/base.in

 edx-django-utils
-edx-django-release-util


Do you know what this is for?

timmc-edx · 2025-01-27T14:05:44Z

README.rst

@@ -0,0 +1,145 @@
+codejail_service


Various notes from Slack:

... being exposed to the outside world (cors, csrf).

TODO: Reminder to document this and to communicate this to SRE.

Add a blank DB for tests.

[question] May not be worth it, but is this something we can clean-up later? Seems a little risky that something like use of a waffle flag could make it through a unit test. Maybe we create a separate time-boxed ticket?

Remove unneeded requirements files. We're going to deploy the base layer instead of the production layer, and requirements.txt is just not needed at all as far as I can tell.

Can you help me understand this? Is this an issue unique to this new service, or does cookie-cutter need to change? Other?

Remove Dockerfile and devstack.py as these will be provided elsewhere.

I wonder where we (the community) are with this? I thought they wanted some basics. Should we find out, or should we err on removing what we won't use and having the maintainer add it when and if it is needed? It'll still be in git commit history.

Re-enable various edx-django-utils middleware as soon as I can determine how to get them working without a DB. (Some of them use waffle flags.)

Can we task this in an entirely separate ticket so we can prioritize against other work? We might want an issue in edx-django-utils, even if we have an experiments ticket. (edited)

timmc-edx · 2025-01-27T14:06:00Z

README.rst

+  This README was auto-generated. Maintainer: please review its contents and
+  update all relevant sections. Instructions to you are marked with
+  "PLACEHOLDER" or "TODO". Update or remove those sections, and remove this
+  note when you are done.


Still need to get the README filled out properly.

feat: Apply cookiecutter output

391a786

- Use cookiecutter version of .gitignore - Don't include LICENSE.txt (already have LICENSE)

timmc-edx mentioned this pull request Jan 23, 2025

Create Django version of eduNEXT Flask codejail service edx/edx-arch-experiments#894

Open

4 tasks

timmc-edx force-pushed the timmc/init branch from 0b0c84b to eba090b Compare January 24, 2025 00:04

timmc-edx force-pushed the timmc/init branch from eba090b to e0e2474 Compare January 24, 2025 00:07

robrap reviewed Jan 24, 2025

View reviewed changes

timmc-edx commented Jan 27, 2025

View reviewed changes

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Initialize codejail_service application #2

Initialize codejail_service application #2

timmc-edx commented Jan 23, 2025 •

edited

Loading

robrap left a comment

robrap Jan 24, 2025

robrap Jan 24, 2025

robrap Jan 24, 2025

robrap Jan 24, 2025

robrap Jan 24, 2025

robrap Jan 24, 2025

robrap Jan 24, 2025

robrap Jan 24, 2025

robrap Jan 24, 2025

robrap Jan 24, 2025

timmc-edx Jan 27, 2025

timmc-edx Jan 27, 2025

		re_path(r'^api/', include(api_urls)),
		re_path(r'^api-docs/', get_swagger_view(title='codejail-service API')),

		edx-django-utils
		edx-django-release-util

Initialize codejail_service application #2

Are you sure you want to change the base?

Initialize codejail_service application #2

Conversation

timmc-edx commented Jan 23, 2025 • edited Loading

robrap left a comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

timmc-edx commented Jan 23, 2025 •

edited

Loading