Skip to content

Commit

Permalink
Merge pull request #253 from lookit/django-docs-update
Browse files Browse the repository at this point in the history 
New local environment directions
  • Loading branch information
@mekline
mekline authored Jul 18, 2022
2 parents 86d2eca + bdd311b commit 3411458
Showing 1 changed file with 49 additions and 109 deletions.
158 changes: 49 additions & 109 deletions docs/source/install-django-project.rst
View file
Original file line number Diff line number Diff line change
Expand Up @@ -22,161 +22,120 @@ downloading data as a researcher.
PR to the lookit-api repo to update the documentation! For notes on Linux installation,
there may be helpful information in a `previous version of invoke tasks.py <https://github.com/lookit/lookit-api/blob/d1b8c9b43cb7d7bda7cdbe5958236d99af42341d/tasks.py>`__.

Django
~~~~~~

These instructions will get walk you through the installation and running of the Django's local envionment.

#. Install `Homebrew <https://brew.sh/>`__. Run update and upgrade:

.. code-block:: shell
Prerequisites
~~~~~~~~~~~~~

brew update && brew upgrade
Before you can begin setting your local environment, we'll need to install a couple of things.

#. Install `Docker <https://docs.docker.com/docker-for-mac/install/>`__ and make sure it's running.
#. Install `mkcert <https://github.com/FiloSottile/mkcert>`__.

#. Install Git and Poetry:
#. Install `Docker Desktop <https://www.docker.com/products/docker-desktop/>`__.

.. code-block:: shell
Local Environment
~~~~~~~~~~~~~~~~~

brew install git poetry
Now the Prerequisites have been installed and Docker is running we can setup the local environment.

#. Change directory to where you want the project to live. Next, clone repo and change directory.
#. Clone lookit-api and change directory:

.. code-block:: shell
git clone [email protected]:lookit/lookit-api.git
cd lookit-api
#. Set Poetry's Python and install Python packages:
#. Create environment file:

.. code-block:: shell
cp env_dist .env
poetry env use python3.9
poetry run pip -U pip wheel setuptools
poetry install
#. Create database and run setup script:
#. Make local CA and certificates for HTTPS:

.. code-block:: shell
poetry run invoke create-db setup
This will create a local .env file with environment variables for local development,
run the Django application's database migrations ("catching up" on changes to the
database structure) and create local SSL certificates. If you're curious about what
exactly is happening during this step, or run into any problems, you can reference the
file `tasks.py <https://github.com/lookit/lookit-api/blob/develop/tasks.py>`__.
make local-certs
#. Run local environment server:
#. Run DB migrations and add entry into Site table:

.. code-block:: shell
poetry run ./manage.py runserver
make migrate
make site
#. Navigate to local environment at http://localhost:8000.

Rabbitmq
~~~~~~~~
#. Celery worker will need some permissions setup to run correctly. To setup these permissions, we'll need to first start the Docker services:

These instructions will have you create and start a RabbitMQ image in Docker.

Install and start rabbitmq via docker:
.. code-block:: shell
.. code-block:: shell
make serve
docker run -d --name lookit-rabbit --env-file .env -p 5672:5672 rabbitmq:3.8.16-management-alpine
docker cp ./rabbitmq.sh lookit-rabbit:/rabbitmq.sh
docker exec -it lookit-rabbit /bin/sh -c "sh /rabbitmq.sh"
Once the services are up and worker has exited due to a permission constraint, we'll set the permissions in the container and restart the worker:

Postgres
~~~~~~~~
.. code-block:: shell
This is covered above, but as sometimes databases can be ephemeral during development I felt that it deserved its own section.
make broker-perms
Create a Postgres database using the following command:

.. code-block:: shell
From time to time, the container will need to be recreated, when this happens you may need to run "broker-perms" again.

poetry run create-db
At this point, the services should all be up and running. Direct your browser to https://localhost:8000 to see the local environment. In the future, to start the services you will only need to run "make serve"

To reset the database:

.. code-block:: shell
Django
~~~~~~

poetry run reset-db
Here are few Django related tasks that might come up every now and then.

To reset the database and load an existing sql data file:
To migrate the existing database:

.. code-block:: shell
poetry run reset-db -s /location/of/sql/file
make migrate
Create Superuser
~~~~~~~~~~~~~~~~

You can create a user through UI or if you'd rather have Superuser access you can create a user with the manage script.

Create a superuser by running:
If you need to create an entry in the Site table:

.. code-block:: shell
poetry run ./manage.py createsuperuser
Now you should be ready for anything. Going forward, you can run the server using the
directions below.

Running the server
~~~~~~~~~~~~~~~~~~~

Again, this is covered above, but there is a case you'd need to run the development server with SSL. This section will cover both variants.
make site
To run the Lookit server locally, run:
To create a superuser:

.. code-block:: shell
poetry run ./manage.py runserver
make superuser
Or to use the invoke script:
To run tests:

.. code-block:: shell
poetry run invoke server
make tests
Now you can go to http://localhost:8000 to see your local Lookit server! You should be able to log in using
the superuser credentials you created during setup.
To view the HTTPS version of the local development add the ``https`` argument to the above command:
Rabbitmq
~~~~~~~~

The broker should come up with the rest of the Docker services. If you get a Celery worker error due to permissions, you can run the following command to resolve the issue and restart the worker service:

.. code-block:: shell
poetry run invoke server --https
make broker-perms
If you are not working extensively with lookit-api - i.e., if you just want to make some
new frames - you do not need to run celery, rabbitmq, or docker. For more information about
these services and how they interact, please see the `Contributing guidelines <https://github.com/lookit/lookit-api/blob/develop/CONTRIBUTING.md>`__.
Postgres
~~~~~~~~

Celery
~~~~~~~~~~~~~~
Here are a couple of command that might be useful for managing the local database.

You should already have a rabbitmq server installed and running. You can check this by:
To access the database shell:

.. code-block:: shell
docker ps -f name=lookit-rabbit
If rabbitmq is not running, you can start it using:

.. code-block:: shell
make dbshell
docker start lookit-rabbit
Then use the invoke command to start the celery worker:
To import a SQL file into a fresh database (one where migrations haven't been ran):

.. code-block:: shell
poetry run invoke celery-service
cat /location/of/sql/file | make dbpipe
Authentication
~~~~~~~~~~~~~~
Expand Down Expand Up @@ -208,22 +167,3 @@ during testing:
Then, based on the the assigned URL, you will need to manually edit the webhook on the
dev environment of Pipe to send the ``video_copied_s3`` event to (for example)
``https://8b48ad70.ngrok.io/exp/renamevideo/``.


Common Issues
~~~~~~~~~~~~~

During installation, you may see the following:

::

psql: FATAL: role "postgres" does not exist

To fix, run something like the following from your home directory:

::

$../../../usr/local/Cellar/postgresql/9.6.3/bin/createuser -s postgres

If your version of postgres is different than 9.6.3, replace with the
correct version above. Running this command should be a one-time thing.

0 comments on commit 3411458

Please sign in to comment.