From bdd311bd34d0289b46046a025ee09136c60aec9b Mon Sep 17 00:00:00 2001 From: CJ Green <44074998+okaycj@users.noreply.github.com> Date: Tue, 21 Jun 2022 15:53:35 -0400 Subject: [PATCH] New local environment directions --- docs/source/install-django-project.rst | 158 ++++++++----------------- 1 file changed, 49 insertions(+), 109 deletions(-) diff --git a/docs/source/install-django-project.rst b/docs/source/install-django-project.rst index 77fc5171..f5d503d6 100644 --- a/docs/source/install-django-project.rst +++ b/docs/source/install-django-project.rst @@ -22,161 +22,120 @@ downloading data as a researcher. PR to the lookit-api repo to update the documentation! For notes on Linux installation, there may be helpful information in a `previous version of invoke tasks.py `__. -Django -~~~~~~ - -These instructions will get walk you through the installation and running of the Django's local envionment. - -#. Install `Homebrew `__. Run update and upgrade: - .. code-block:: shell +Prerequisites +~~~~~~~~~~~~~ - brew update && brew upgrade +Before you can begin setting your local environment, we'll need to install a couple of things. -#. Install `Docker `__ and make sure it's running. +#. Install `mkcert `__. -#. Install Git and Poetry: +#. Install `Docker Desktop `__. - .. code-block:: shell +Local Environment +~~~~~~~~~~~~~~~~~ - brew install git poetry +Now the Prerequisites have been installed and Docker is running we can setup the local environment. -#. Change directory to where you want the project to live. Next, clone repo and change directory. +#. Clone lookit-api and change directory: .. code-block:: shell git clone git@github.com:lookit/lookit-api.git cd lookit-api -#. Set Poetry's Python and install Python packages: +#. Create environment file: .. code-block:: shell + + cp env_dist .env - poetry env use python3.9 - poetry run pip -U pip wheel setuptools - poetry install - -#. Create database and run setup script: +#. Make local CA and certificates for HTTPS: .. code-block:: shell - poetry run invoke create-db setup - - This will create a local .env file with environment variables for local development, - run the Django application's database migrations ("catching up" on changes to the - database structure) and create local SSL certificates. If you're curious about what - exactly is happening during this step, or run into any problems, you can reference the - file `tasks.py `__. + make local-certs -#. Run local environment server: +#. Run DB migrations and add entry into Site table: .. code-block:: shell - poetry run ./manage.py runserver + make migrate + make site -#. Navigate to local environment at http://localhost:8000. - -Rabbitmq -~~~~~~~~ +#. Celery worker will need some permissions setup to run correctly. To setup these permissions, we'll need to first start the Docker services: -These instructions will have you create and start a RabbitMQ image in Docker. - -Install and start rabbitmq via docker: + .. code-block:: shell -.. code-block:: shell + make serve - docker run -d --name lookit-rabbit --env-file .env -p 5672:5672 rabbitmq:3.8.16-management-alpine - docker cp ./rabbitmq.sh lookit-rabbit:/rabbitmq.sh - docker exec -it lookit-rabbit /bin/sh -c "sh /rabbitmq.sh" + Once the services are up and worker has exited due to a permission constraint, we'll set the permissions in the container and restart the worker: -Postgres -~~~~~~~~ + .. code-block:: shell -This is covered above, but as sometimes databases can be ephemeral during development I felt that it deserved its own section. + make broker-perms -Create a Postgres database using the following command: - -.. code-block:: shell + From time to time, the container will need to be recreated, when this happens you may need to run "broker-perms" again. - poetry run create-db +At this point, the services should all be up and running. Direct your browser to https://localhost:8000 to see the local environment. In the future, to start the services you will only need to run "make serve" -To reset the database: -.. code-block:: shell +Django +~~~~~~ - poetry run reset-db +Here are few Django related tasks that might come up every now and then. -To reset the database and load an existing sql data file: +To migrate the existing database: .. code-block:: shell - poetry run reset-db -s /location/of/sql/file - + make migrate -Create Superuser -~~~~~~~~~~~~~~~~ - -You can create a user through UI or if you'd rather have Superuser access you can create a user with the manage script. - -Create a superuser by running: +If you need to create an entry in the Site table: .. code-block:: shell - poetry run ./manage.py createsuperuser - -Now you should be ready for anything. Going forward, you can run the server using the -directions below. - -Running the server -~~~~~~~~~~~~~~~~~~~ - -Again, this is covered above, but there is a case you'd need to run the development server with SSL. This section will cover both variants. + make site -To run the Lookit server locally, run: +To create a superuser: .. code-block:: shell - poetry run ./manage.py runserver + make superuser -Or to use the invoke script: +To run tests: .. code-block:: shell - poetry run invoke server + make tests -Now you can go to http://localhost:8000 to see your local Lookit server! You should be able to log in using -the superuser credentials you created during setup. -To view the HTTPS version of the local development add the ``https`` argument to the above command: +Rabbitmq +~~~~~~~~ + +The broker should come up with the rest of the Docker services. If you get a Celery worker error due to permissions, you can run the following command to resolve the issue and restart the worker service: .. code-block:: shell - poetry run invoke server --https + make broker-perms -If you are not working extensively with lookit-api - i.e., if you just want to make some -new frames - you do not need to run celery, rabbitmq, or docker. For more information about -these services and how they interact, please see the `Contributing guidelines `__. +Postgres +~~~~~~~~ -Celery -~~~~~~~~~~~~~~ +Here are a couple of command that might be useful for managing the local database. -You should already have a rabbitmq server installed and running. You can check this by: +To access the database shell: .. code-block:: shell - docker ps -f name=lookit-rabbit - -If rabbitmq is not running, you can start it using: - -.. code-block:: shell + make dbshell - docker start lookit-rabbit - -Then use the invoke command to start the celery worker: +To import a SQL file into a fresh database (one where migrations haven't been ran): .. code-block:: shell - poetry run invoke celery-service + cat /location/of/sql/file | make dbpipe + Authentication ~~~~~~~~~~~~~~ @@ -208,22 +167,3 @@ during testing: Then, based on the the assigned URL, you will need to manually edit the webhook on the dev environment of Pipe to send the ``video_copied_s3`` event to (for example) ``https://8b48ad70.ngrok.io/exp/renamevideo/``. - - -Common Issues -~~~~~~~~~~~~~ - -During installation, you may see the following: - -:: - - psql: FATAL: role "postgres" does not exist - -To fix, run something like the following from your home directory: - -:: - - $../../../usr/local/Cellar/postgresql/9.6.3/bin/createuser -s postgres - -If your version of postgres is different than 9.6.3, replace with the -correct version above. Running this command should be a one-time thing.