-
Notifications
You must be signed in to change notification settings - Fork 16
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Import shapes with new management command #256
Conversation
data/scripts/extract_shapes.py
Outdated
def get_response(url, params=None, timeout=60, **kwargs): | ||
""" | ||
The OCD API has intermittently thrown 502 and 504 errors, so only proceed | ||
when receiving an 'ok' status. |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
I adapted this function from https://github.com/datamade/django-councilmatic/blob/5387ae85ea289b22c4e0ea566fa89a1c43155ed0/councilmatic_core/management/commands/import_data.py#L3250-L3261, including the docstring. I'm not actually sure whether the docstring accurately represents what's going on in the function -- it seems to me that the method is simply a thin wrapper around requests
to make sure that it A) uses a session and B) raises errors from non-200 status codes -- but the docstring seems to perhaps be communicating an aspect of the OCD API that is useful to know.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
I think we don't want this. Let's just get https://data.cityofchicago.org/Facilities-Geographic-Boundaries/Boundaries-Wards-2015-/sp34-6z76 and get into the right shape (see discussion in the other PR)
data/scripts/extract_shapes.py
Outdated
def get_response(url, params=None, timeout=60, **kwargs): | ||
""" | ||
The OCD API has intermittently thrown 502 and 504 errors, so only proceed | ||
when receiving an 'ok' status. |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
I think we don't want this. Let's just get https://data.cityofchicago.org/Facilities-Geographic-Boundaries/Boundaries-Wards-2015-/sp34-6z76 and get into the right shape (see discussion in the other PR)
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
thanks for bearing with me Jean
Overview
Following the changes in datamade/django-councilmatic#249, create a
data
directory to store Post shapes locally, and use the newimport_shapes
management command to load those shapes into the database.Testing instructions
Test the Makefile
cd data
make clean
make
Test the
import_shapes
commanddjango-councilmatic
instance, and make sure it's installed to thechi-councilmatic
virtualenv withpip install -e /path/to/django-councilmatic
python manage.py migrate
python manage.py import_shapes data/final/shapes/chicago_shapes.json
python manage.py runserver