[Temp] Local testing documentation

[anayeaye]

What

This issue is a place to stash notes about testing the backend APIs, in particular to support testing endpoints that require custom database migrations not included in the dockerfile and test data not yet included in the project.

Running locally

Install requirements

python3 -m pip install --upgrade pip
python3 -m pip install -e ".[dev,test]"

Start up a local stack

docker compose up

# Clean up when done running
docker compose down

The local stac-api should be running on 8081 and the raster-api should be running on 8082 (you can confirm these ports in docker-compose.yml).

Loading data

With your app running locally, ingest some STAC records following the github actions test workflow in .github/workflows/pr.yml.

Additional test collections and items can be loaded using the same pypgstac commands as the github actions, for example:

pypgstac load collections .github/workflows/data/limited-spatial-extent-collection.json  --dsn postgresql://username:[email protected]:5432/postgis --method insert.

A collection with 0 items

itemless-collection.json

{"id":"itemless-collection", "title": "Test collection without items", "description":"An itemless collection.","stac_version":"1.0.0","license":"public-domain","links":[],"extent":{"spatial":{"bbox":[[-180,-90,180,90]]},"temporal":{"interval":[["2005-01-01T00:00:00Z","null"]]}}}

Subset collection

A subset limited-spatial-extent-collection with a smaller spatial and temporal extent than the noaa-emergency-response collection

limited-spatial-extent-collection.json

{"id":"limited-spatial-extent-collection", "title": "Collection with small spatial extent", "description":"Test collection with small spatial extent.","stac_version":"1.0.0","license":"public-domain","links":[],"extent":{"spatial":{"bbox":[[-86.9501, 36.1249, -86.4499, 36.2251]]},"temporal":{"interval":[["2020-03-10T00:00:00Z","2020-03-11T00:00:00Z"]]}}}

limited-spatial-extent-items.json

{"type":"Feature","stac_version":"1.0.0","id":"20200307aC0862830w361330","properties":{"event":"Nashville Tornado","datetime":"2020-03-10T00:00:00Z"},"geometry":{"type":"Polygon","coordinates":[[[-86.4751,36.2251],[-86.4751,36.1999],[-86.4499,36.1999],[-86.4499,36.2251],[-86.4751,36.2251]]]},"assets":{"cog":{"href":"https://noaa-eri-pds.s3.us-east-1.amazonaws.com/2020_Nashville_Tornado/20200307a_RGB/20200307aC0862830w361330n.tif","type":"image/tiff; application=geotiff; profile=cloud-optimized"}},"bbox":[-86.4751,36.1999,-86.4499,36.2251],"stac_extensions":[],"collection":"limited-spatial-extent-collection"}
{"type":"Feature","stac_version":"1.0.0","id":"20200307aC0863130w361200","properties":{"event":"Nashville Tornado","datetime":"2020-03-10T00:00:00Z"},"geometry":{"type":"Polygon","coordinates":[[[-86.5251,36.2001],[-86.5251,36.1749],[-86.4999,36.1749],[-86.4999,36.2001],[-86.5251,36.2001]]]},"assets":{"cog":{"href":"https://noaa-eri-pds.s3.us-east-1.amazonaws.com/2020_Nashville_Tornado/20200307a_RGB/20200307aC0863130w361200n.tif","type":"image/tiff; application=geotiff; profile=cloud-optimized"}},"bbox":[-86.5251,36.1749,-86.4999,36.2001],"stac_extensions":[],"collection":"limited-spatial-extent-collection"}
{"type":"Feature","stac_version":"1.0.0","id":"20200307aC0863130w361330","properties":{"event":"Nashville Tornado","datetime":"2020-03-11T00:00:00Z"},"geometry":{"type":"Polygon","coordinates":[[[-86.5251,36.2251],[-86.5251,36.1999],[-86.4999,36.1999],[-86.4999,36.2251],[-86.5251,36.2251]]]},"assets":{"cog":{"href":"https://noaa-eri-pds.s3.us-east-1.amazonaws.com/2020_Nashville_Tornado/20200307a_RGB/20200307aC0863130w361330n.tif","type":"image/tiff; application=geotiff; profile=cloud-optimized"}},"bbox":[-86.5251,36.1999,-86.4999,36.2251],"stac_extensions":[],"collection":"limited-spatial-extent-collection"}
{"type":"Feature","stac_version":"1.0.0","id":"20200307aC0863430w361030","properties":{"event":"Nashville Tornado","datetime":"2020-03-10T00:00:00Z"},"geometry":{"type":"Polygon","coordinates":[[[-86.5751,36.1751],[-86.5751,36.1499],[-86.5499,36.1499],[-86.5499,36.1751],[-86.5751,36.1751]]]},"assets":{"cog":{"href":"https://noaa-eri-pds.s3.us-east-1.amazonaws.com/2020_Nashville_Tornado/20200307a_RGB/20200307aC0863430w361030n.tif","type":"image/tiff; application=geotiff; profile=cloud-optimized"}},"bbox":[-86.5751,36.1499,-86.5499,36.1751],"stac_extensions":[],"collection":"limited-spatial-extent-collection"}
{"type":"Feature","stac_version":"1.0.0","id":"20200307aC0863430w361200","properties":{"event":"Nashville Tornado","datetime":"2020-03-11T00:00:00Z"},"geometry":{"type":"Polygon","coordinates":[[[-86.5751,36.2001],[-86.5751,36.1749],[-86.5499,36.1749],[-86.5499,36.2001],[-86.5751,36.2001]]]},"assets":{"cog":{"href":"https://noaa-eri-pds.s3.us-east-1.amazonaws.com/2020_Nashville_Tornado/20200307a_RGB/20200307aC0863430w361200n.tif","type":"image/tiff; application=geotiff; profile=cloud-optimized"}},"bbox":[-86.5751,36.1749,-86.5499,36.2001],"stac_extensions":[],"collection":"limited-spatial-extent-collection"}
{"type":"Feature","stac_version":"1.0.0","id":"20200307aC0863730w361030","properties":{"event":"Nashville Tornado","datetime":"2020-03-10T00:00:00Z"},"geometry":{"type":"Polygon","coordinates":[[[-86.6251,36.1751],[-86.6251,36.1499],[-86.5999,36.1499],[-86.5999,36.1751],[-86.6251,36.1751]]]},"assets":{"cog":{"href":"https://noaa-eri-pds.s3.us-east-1.amazonaws.com/2020_Nashville_Tornado/20200307a_RGB/20200307aC0863730w361030n.tif","type":"image/tiff; application=geotiff; profile=cloud-optimized"}},"bbox":[-86.6251,36.1499,-86.5999,36.1751],"stac_extensions":[],"collection":"limited-spatial-extent-collection"}
{"type":"Feature","stac_version":"1.0.0","id":"20200307aC0863730w361200","properties":{"event":"Nashville Tornado","datetime":"2020-03-11T00:00:00Z"},"geometry":{"type":"Polygon","coordinates":[[[-86.6251,36.2001],[-86.6251,36.1749],[-86.5999,36.1749],[-86.5999,36.2001],[-86.6251,36.2001]]]},"assets":{"cog":{"href":"https://noaa-eri-pds.s3.us-east-1.amazonaws.com/2020_Nashville_Tornado/20200307a_RGB/20200307aC0863730w361200n.tif","type":"image/tiff; application=geotiff; profile=cloud-optimized"}},"bbox":[-86.6251,36.1749,-86.5999,36.2001],"stac_extensions":[],"collection":"limited-spatial-extent-collection"}
{"type":"Feature","stac_version":"1.0.0","id":"20200307aC0863900w361330","properties":{"event":"Nashville Tornado","datetime":"2020-03-10T00:00:00Z"},"geometry":{"type":"Polygon","coordinates":[[[-86.6501,36.2251],[-86.6501,36.1999],[-86.6249,36.1999],[-86.6249,36.2251],[-86.6501,36.2251]]]},"assets":{"cog":{"href":"https://noaa-eri-pds.s3.us-east-1.amazonaws.com/2020_Nashville_Tornado/20200307a_RGB/20200307aC0863900w361330n.tif","type":"image/tiff; application=geotiff; profile=cloud-optimized"}},"bbox":[-86.6501,36.1999,-86.6249,36.2251],"stac_extensions":[],"collection":"limited-spatial-extent-collection"}
{"type":"Feature","stac_version":"1.0.0","id":"20200307aC0864030w361030","properties":{"event":"Nashville Tornado","datetime":"2020-03-11T00:00:00Z"},"geometry":{"type":"Polygon","coordinates":[[[-86.6751,36.1751],[-86.6751,36.1499],[-86.6499,36.1499],[-86.6499,36.1751],[-86.6751,36.1751]]]},"assets":{"cog":{"href":"https://noaa-eri-pds.s3.us-east-1.amazonaws.com/2020_Nashville_Tornado/20200307a_RGB/20200307aC0864030w361030n.tif","type":"image/tiff; application=geotiff; profile=cloud-optimized"}},"bbox":[-86.6751,36.1499,-86.6499,36.1751],"stac_extensions":[],"collection":"limited-spatial-extent-collection"}
{"type":"Feature","stac_version":"1.0.0","id":"20200307aC0864200w361200","properties":{"event":"Nashville Tornado","datetime":"2020-03-10T00:00:00Z"},"geometry":{"type":"Polygon","coordinates":[[[-86.7001,36.2001],[-86.7001,36.1749],[-86.6749,36.1749],[-86.6749,36.2001],[-86.7001,36.2001]]]},"assets":{"cog":{"href":"https://noaa-eri-pds.s3.us-east-1.amazonaws.com/2020_Nashville_Tornado/20200307a_RGB/20200307aC0864200w361200n.tif","type":"image/tiff; application=geotiff; profile=cloud-optimized"}},"bbox":[-86.7001,36.1749,-86.6749,36.2001],"stac_extensions":[],"collection":"limited-spatial-extent-collection"}
{"type":"Feature","stac_version":"1.0.0","id":"20200307aC0864330w361030","properties":{"event":"Nashville Tornado","datetime":"2020-03-11T00:00:00Z"},"geometry":{"type":"Polygon","coordinates":[[[-86.7251,36.1751],[-86.7251,36.1499],[-86.6999,36.1499],[-86.6999,36.1751],[-86.7251,36.1751]]]},"assets":{"cog":{"href":"https://noaa-eri-pds.s3.us-east-1.amazonaws.com/2020_Nashville_Tornado/20200307a_RGB/20200307aC0864330w361030n.tif","type":"image/tiff; application=geotiff; profile=cloud-optimized"}},"bbox":[-86.7251,36.1499,-86.6999,36.1751],"stac_extensions":[],"collection":"limited-spatial-extent-collection"}
{"type":"Feature","stac_version":"1.0.0","id":"20200307aC0864630w361030","properties":{"event":"Nashville Tornado","datetime":"2020-03-10T00:00:00Z"},"geometry":{"type":"Polygon","coordinates":[[[-86.7751,36.1751],[-86.7751,36.1499],[-86.7499,36.1499],[-86.7499,36.1751],[-86.7751,36.1751]]]},"assets":{"cog":{"href":"https://noaa-eri-pds.s3.us-east-1.amazonaws.com/2020_Nashville_Tornado/20200307a_RGB/20200307aC0864630w361030n.tif","type":"image/tiff; application=geotiff; profile=cloud-optimized"}},"bbox":[-86.7751,36.1499,-86.7499,36.1751],"stac_extensions":[],"collection":"limited-spatial-extent-collection"}
{"type":"Feature","stac_version":"1.0.0","id":"20200307aC0864630w361200","properties":{"event":"Nashville Tornado","datetime":"2020-03-11T00:00:00Z"},"geometry":{"type":"Polygon","coordinates":[[[-86.7751,36.2001],[-86.7751,36.1749],[-86.7499,36.1749],[-86.7499,36.2001],[-86.7751,36.2001]]]},"assets":{"cog":{"href":"https://noaa-eri-pds.s3.us-east-1.amazonaws.com/2020_Nashville_Tornado/20200307a_RGB/20200307aC0864630w361200n.tif","type":"image/tiff; application=geotiff; profile=cloud-optimized"}},"bbox":[-86.7751,36.1749,-86.7499,36.2001],"stac_extensions":[],"collection":"limited-spatial-extent-collection"}
{"type":"Feature","stac_version":"1.0.0","id":"20200307aC0865100w361030","properties":{"event":"Nashville Tornado","datetime":"2020-03-10T00:00:00Z"},"geometry":{"type":"Polygon","coordinates":[[[-86.8501,36.1751],[-86.8501,36.1499],[-86.8249,36.1499],[-86.8249,36.1751],[-86.8501,36.1751]]]},"assets":{"cog":{"href":"https://noaa-eri-pds.s3.us-east-1.amazonaws.com/2020_Nashville_Tornado/20200307a_RGB/20200307aC0865100w361030n.tif","type":"image/tiff; application=geotiff; profile=cloud-optimized"}},"bbox":[-86.8501,36.1499,-86.8249,36.1751],"stac_extensions":[],"collection":"limited-spatial-extent-collection"}
{"type":"Feature","stac_version":"1.0.0","id":"20200307aC0865100w361200","properties":{"event":"Nashville Tornado","datetime":"2020-03-11T00:00:00Z"},"geometry":{"type":"Polygon","coordinates":[[[-86.8501,36.2001],[-86.8501,36.1749],[-86.8249,36.1749],[-86.8249,36.2001],[-86.8501,36.2001]]]},"assets":{"cog":{"href":"https://noaa-eri-pds.s3.us-east-1.amazonaws.com/2020_Nashville_Tornado/20200307a_RGB/20200307aC0865100w361200n.tif","type":"image/tiff; application=geotiff; profile=cloud-optimized"}},"bbox":[-86.8501,36.1749,-86.8249,36.2001],"stac_extensions":[],"collection":"limited-spatial-extent-collection"}
{"type":"Feature","stac_version":"1.0.0","id":"20200307aC0865400w361200","properties":{"event":"Nashville Tornado","datetime":"2020-03-10T00:00:00Z"},"geometry":{"type":"Polygon","coordinates":[[[-86.9001,36.2001],[-86.9001,36.1749],[-86.8749,36.1749],[-86.8749,36.2001],[-86.9001,36.2001]]]},"assets":{"cog":{"href":"https://noaa-eri-pds.s3.us-east-1.amazonaws.com/2020_Nashville_Tornado/20200307a_RGB/20200307aC0865400w361200n.tif","type":"image/tiff; application=geotiff; profile=cloud-optimized"}},"bbox":[-86.9001,36.1749,-86.8749,36.2001],"stac_extensions":[],"collection":"limited-spatial-extent-collection"}
{"type":"Feature","stac_version":"1.0.0","id":"20200307aC0865530w360900","properties":{"event":"Nashville Tornado","datetime":"2020-03-11T00:00:00Z"},"geometry":{"type":"Polygon","coordinates":[[[-86.9251,36.1501],[-86.9251,36.1249],[-86.8999,36.1249],[-86.8999,36.1501],[-86.9251,36.1501]]]},"assets":{"cog":{"href":"https://noaa-eri-pds.s3.us-east-1.amazonaws.com/2020_Nashville_Tornado/20200307a_RGB/20200307aC0865530w360900n.tif","type":"image/tiff; application=geotiff; profile=cloud-optimized"}},"bbox":[-86.9251,36.1249,-86.8999,36.1501],"stac_extensions":[],"collection":"limited-spatial-extent-collection"}
{"type":"Feature","stac_version":"1.0.0","id":"20200307aC0865700w361030","properties":{"event":"Nashville Tornado","datetime":"2020-03-10T00:00:00Z"},"geometry":{"type":"Polygon","coordinates":[[[-86.9501,36.1751],[-86.9501,36.1499],[-86.9249,36.1499],[-86.9249,36.1751],[-86.9501,36.1751]]]},"assets":{"cog":{"href":"https://noaa-eri-pds.s3.us-east-1.amazonaws.com/2020_Nashville_Tornado/20200307a_RGB/20200307aC0865700w361030n.tif","type":"image/tiff; application=geotiff; profile=cloud-optimized"}},"bbox":[-86.9501,36.1499,-86.9249,36.1751],"stac_extensions":[],"collection":"limited-spatial-extent-collection"}
{"type":"Feature","stac_version":"1.0.0","id":"20200307aC0865700w361200","properties":{"event":"Nashville Tornado","datetime":"2020-03-11T00:00:00Z"},"geometry":{"type":"Polygon","coordinates":[[[-86.9501,36.2001],[-86.9501,36.1749],[-86.9249,36.1749],[-86.9249,36.2001],[-86.9501,36.2001]]]},"assets":{"cog":{"href":"https://noaa-eri-pds.s3.us-east-1.amazonaws.com/2020_Nashville_Tornado/20200307a_RGB/20200307aC0865700w361200n.tif","type":"image/tiff; application=geotiff; profile=cloud-optimized"}},"bbox":[-86.9501,36.1749,-86.9249,36.2001],"stac_extensions":[],"collection":"limited-spatial-extent-collection"}

Run tests

With the local stack running and data in postgres, the tests in .github/workflows/tests can be run with pytest.

Sample requests

These example requests require loading the noaa-emergency-response and limited-spatial-extent-collection collections and corresponding items. The examples are for the collection-id-search endpoint which is not yet tested in the github workflows. The request format and queries should be interchangeable with the item /search endpoint [POST].

Collection-id-search no filters

All collections with items

curl -X 'POST' \
  'http://localhost:8081/collection-id-search' \
  -H 'accept: application/json' \
  -H 'Content-Type: application/json' \
  -d '{}'

expected out >
[
  "noaa-emergency-response",
  "limited-spatial-extent-collection"
]

Collection-id-search temporal filter

One collection excluded by datetime filter

curl -X 'POST' \
  'http://localhost:8081/collection-id-search' \
  -H 'accept: application/json' \
  -H 'Content-Type: application/json' \
  -d '{"datetime":"2020-03-07T00:00:00Z/2020-03-08T00:00:00Z"}'

expected out >
[
  "noaa-emergency-response"
]

Collection-id-search spatial filter

Spatial filter excludes one collection

curl -X 'POST' \
  'http://localhost:8081/collection-id-search' \
  -H 'accept: application/json' \
  -H 'Content-Type: application/json' \
  -d '{"intersects":{"type":"Polygon","coordinates":[[[-85.8251,36.2001],[-85.8251,36.1749],[-85.7999,36.1749],[-85.7999,36.2001],[-85.8251,36.2001]]]}}'

expected out >
[
  "noaa-emergency-response"
]