Adding route dependencies module. (#251)

**Description:** Generate a list of route dependencies from the environment variable `STAC_FASTAPI_ROUTE_DEPENDENCIES` or the file that this environment variable points to. Feed these route dependencies into the `STACApi` on initialisation. **PR Checklist:** - [x] Code is formatted and linted (run `pre-commit run --all-files`) - [x] Tests pass (run `make test`) - [x] Documentation has been updated to reflect changes, if applicable - [x] Changes are added to the changelog --------- Co-authored-by: Jonathan Healy <[email protected]>
stac-utils · Jun 22, 2024 · de78fbc · de78fbc
1 parent f4618a9
commit de78fbc
Show file tree

Hide file tree

Showing 16 changed files with 2,499 additions and 203 deletions.
diff --git a/CHANGELOG.md b/CHANGELOG.md
@@ -7,6 +7,15 @@ and this project adheres to [Semantic Versioning](http://semver.org/spec/v2.0.0.
 
 ## [Unreleased]
 
+### Added
+ - Added support for route depndencies configuration through the STAC_FASTAPI_ROUTE_DEPENDENCIES environment variable, directly or via json file. Allows for fastapi's inbuilt OAuth2 flows to be used as dependencies. Custom dependencies can also be written, see Basic Auth for an example. [#251](https://github.com/stac-utils/stac-fastapi-elasticsearch-opensearch/pull/251)
+ - Added docker-compose.route_dependencies_file.yml that gives an example of OAuth2 workflow using keycloak as the identity provider. [#251](https://github.com/stac-utils/stac-fastapi-elasticsearch-opensearch/pull/251)
+ - Added docker-compose.route_dependencies_env.yml that gives an example using the STAC_FASTAPI_ROUTE_DEPENDENCIES environment variable. [#251](https://github.com/stac-utils/stac-fastapi-elasticsearch-opensearch/pull/251)
+
+### Changed
+- Converted Basic auth to a route dependency and merged with new route depndencies method. [#251](https://github.com/stac-utils/stac-fastapi-elasticsearch-opensearch/pull/251)
+- Updated docker-compose.basic_auth_protected.yml to use STAC_FASTAPI_ROUTE_DEPENDENCIES environment variable. [#251](https://github.com/stac-utils/stac-fastapi-elasticsearch-opensearch/pull/251)
+
 ## [v3.0.0a2]
 
 ### Added
@@ -234,4 +243,4 @@ and this project adheres to [Semantic Versioning](http://semver.org/spec/v2.0.0.
 [v1.0.0]: <https://github.com/stac-utils/stac-fastapi-elasticsearch/tree/v0.3.0...v1.0.0>
 [v0.3.0]: <https://github.com/stac-utils/stac-fastapi-elasticsearch/tree/v0.2.0...v0.3.0>
 [v0.2.0]: <https://github.com/stac-utils/stac-fastapi-elasticsearch/tree/v0.1.0...v0.2.0>
-[v0.1.0]: <https://github.com/stac-utils/stac-fastapi-elasticsearch/tree/v0.1.0>
+[v0.1.0]: <https://github.com/stac-utils/stac-fastapi-elasticsearch/tree/v0.1.0>
diff --git a/README.md b/README.md
@@ -279,75 +279,106 @@ The modified Items with lowercase identifiers will now be visible to users acces
 
 #### Environment Variable Configuration
 
-Basic authentication is an optional feature. You can enable it by setting the environment variable `BASIC_AUTH` as a JSON string.
+Basic authentication is an optional feature that can be enabled through [Route Dependencies](#route-dependencies). 
 
-Example:
-```
-BASIC_AUTH={"users":[{"username":"user","password":"pass","permissions":"*"}]}
-```
 
-### User Permissions Configuration
+### Example Configuration
 
-In order to set endpoints with specific access permissions, you can configure the `users` key with a list of user objects. Each user object should contain the username, password, and their respective permissions.
-
-Example: This example illustrates the configuration for two users: an **admin** user with full permissions (*) and a **reader** user with limited permissions to specific read-only endpoints.
+This example illustrates the configuration for two users: an **admin** user with full permissions (*) and a **reader** user with limited permissions to specific read-only endpoints.
 ```json
-{
-    "users": [
-        {
-            "username": "admin",
-            "password": "admin",
-            "permissions": "*"
-        },
-        {
-            "username": "reader",
-            "password": "reader",
-            "permissions": [
-                {"path": "/", "method": ["GET"]},
-                {"path": "/conformance", "method": ["GET"]},
-                {"path": "/collections/{collection_id}/items/{item_id}", "method": ["GET"]},
-                {"path": "/search", "method": ["GET", "POST"]},
-                {"path": "/collections", "method": ["GET"]},
-                {"path": "/collections/{collection_id}", "method": ["GET"]},
-                {"path": "/collections/{collection_id}/items", "method": ["GET"]},
-                {"path": "/queryables", "method": ["GET"]},
-                {"path": "/queryables/collections/{collection_id}/queryables", "method": ["GET"]},
-                {"path": "/_mgmt/ping", "method": ["GET"]}
-            ]
+[
+  {
+    "routes": [
+      {
+        "method": "*",
+        "path": "*"
+      }
+    ],
+    "dependencies": [
+      {
+        "method": "stac_fastapi.core.basic_auth.BasicAuth",
+        "kwargs": {
+          "credentials":[
+            {
+              "username": "admin",
+              "password": "admin"
+            }
+          ]
+        }
+      }
+    ]
+  },
+  {
+    "routes": [
+      {"path": "/", "method": ["GET"]},
+      {"path": "/conformance", "method": ["GET"]},
+      {"path": "/collections/{collection_id}/items/{item_id}", "method": ["GET"]},
+      {"path": "/search", "method": ["GET", "POST"]},
+      {"path": "/collections", "method": ["GET"]},
+      {"path": "/collections/{collection_id}", "method": ["GET"]},
+      {"path": "/collections/{collection_id}/items", "method": ["GET"]},
+      {"path": "/queryables", "method": ["GET"]},
+      {"path": "/queryables/collections/{collection_id}/queryables", "method": ["GET"]},
+      {"path": "/_mgmt/ping", "method": ["GET"]}
+    ],
+    "dependencies": [
+      {
+        "method": "stac_fastapi.core.basic_auth.BasicAuth",
+        "kwargs": {
+          "credentials":[
+            {
+              "username": "reader",
+              "password": "reader"
+            }
+          ]
         }
+      }
     ]
-}
+  }
+]
 ```
 
+## Route Dependencies
 
-### Public Endpoints Configuration
+### Configuration
 
-In order to set endpoints with public access, you can configure the public_endpoints key with a list of endpoint objects. Each endpoint object should specify the path and method of the endpoint.
+Route dependencies for endpoints can enable through the `STAC_FASTAPI_ROUTE_DEPENDENCIES` 
+environment variable as a path to a JSON file or a JSON string.
 
-Example: This example demonstrates the configuration for public endpoints, allowing access without authentication to read-only endpoints.
-```json
-{
-    "public_endpoints": [
-        {"path": "/", "method": "GET"},
-        {"path": "/conformance", "method": "GET"},
-        {"path": "/collections/{collection_id}/items/{item_id}", "method": "GET"},
-        {"path": "/search", "method": "GET"},
-        {"path": "/search", "method": "POST"},
-        {"path": "/collections", "method": "GET"},
-        {"path": "/collections/{collection_id}", "method": "GET"},
-        {"path": "/collections/{collection_id}/items", "method": "GET"},
-        {"path": "/queryables", "method": "GET"},
-        {"path": "/queryables/collections/{collection_id}/queryables", "method": "GET"},
-        {"path": "/_mgmt/ping", "method": "GET"}
+#### Route Dependency
+
+A Route Dependency must include `routes`, a list of at least one [Route](#routes), and `dependencies` a
+list of at least one [Dependency](#dependencies).
+
+#### Routes
+
+A Route must include a `path`, the relative path to the endpoint, and a `method`, the request method of the path.
+
+#### Dependencies
+
+A Dependency must include the `method`, a dot seperated path to the Class or Function, and 
+can include any `args` or `kwargs` for the method.
+
+#### Example
+```
+STAC_FASTAPI_ROUTE_DEPENDENCIES=[
+  {
+    "routes": [
+      {
+        "method": "GET",
+        "path": "/collections"
+      }
     ],
-    "users": [
-        {
-            "username": "admin",
-            "password": "admin",
-            "permissions": "*"
+    "dependencies": [
+      {
+        "method": "fastapi.security.OAuth2PasswordBearer",
+        "kwargs": {
+          "tokenUrl": "token"
         }
+      }
     ]
-}
+  }
+]
 ```
 
 ### Docker Compose Configurations

diff --git a/docker-compose.basic_auth_protected.yml b/docker-compose.basic_auth_protected.yml
@@ -22,7 +22,7 @@ services:
       - ES_USE_SSL=false
       - ES_VERIFY_CERTS=false
       - BACKEND=elasticsearch
-      - BASIC_AUTH={"users":[{"username":"admin","password":"admin","permissions":"*"},{"username":"reader","password":"reader","permissions":[{"path":"/","method":["GET"]},{"path":"/conformance","method":["GET"]},{"path":"/collections/{collection_id}/items/{item_id}","method":["GET"]},{"path":"/search","method":["GET","POST"]},{"path":"/collections","method":["GET"]},{"path":"/collections/{collection_id}","method":["GET"]},{"path":"/collections/{collection_id}/items","method":["GET"]},{"path":"/queryables","method":["GET"]},{"path":"/queryables/collections/{collection_id}/queryables","method":["GET"]},{"path":"/_mgmt/ping","method":["GET"]}]}]}
+      - STAC_FASTAPI_ROUTE_DEPENDENCIES=[{"routes":[{"method":"*","path":"*"}],"dependencies":[{"method":"stac_fastapi.core.basic_auth.BasicAuth","kwargs":{"credentials":[{"username":"admin","password":"admin"}]}}]},{"routes":[{"path":"/","method":["GET"]},{"path":"/conformance","method":["GET"]},{"path":"/collections/{collection_id}/items/{item_id}","method":["GET"]},{"path":"/search","method":["GET","POST"]},{"path":"/collections","method":["GET"]},{"path":"/collections/{collection_id}","method":["GET"]},{"path":"/collections/{collection_id}/items","method":["GET"]},{"path":"/queryables","method":["GET"]},{"path":"/queryables/collections/{collection_id}/queryables","method":["GET"]},{"path":"/_mgmt/ping","method":["GET"]}],"dependencies":[{"method":"stac_fastapi.core.basic_auth.BasicAuth","kwargs":{"credentials":[{"username":"reader","password":"reader"}]}}]}]
     ports:
       - "8080:8080"
     volumes:
@@ -55,7 +55,7 @@ services:
       - ES_USE_SSL=false
       - ES_VERIFY_CERTS=false
       - BACKEND=opensearch
-      - BASIC_AUTH={"users":[{"username":"admin","password":"admin","permissions":"*"},{"username":"reader","password":"reader","permissions":[{"path":"/","method":["GET"]},{"path":"/conformance","method":["GET"]},{"path":"/collections/{collection_id}/items/{item_id}","method":["GET"]},{"path":"/search","method":["GET","POST"]},{"path":"/collections","method":["GET"]},{"path":"/collections/{collection_id}","method":["GET"]},{"path":"/collections/{collection_id}/items","method":["GET"]},{"path":"/queryables","method":["GET"]},{"path":"/queryables/collections/{collection_id}/queryables","method":["GET"]},{"path":"/_mgmt/ping","method":["GET"]}]}]}
+      - STAC_FASTAPI_ROUTE_DEPENDENCIES=[{"routes":[{"method":"*","path":"*"}],"dependencies":[{"method":"stac_fastapi.core.basic_auth.BasicAuth","kwargs":{"credentials":[{"username":"admin","password":"admin"}]}}]},{"routes":[{"path":"/","method":["GET"]},{"path":"/conformance","method":["GET"]},{"path":"/collections/{collection_id}/items/{item_id}","method":["GET"]},{"path":"/search","method":["GET","POST"]},{"path":"/collections","method":["GET"]},{"path":"/collections/{collection_id}","method":["GET"]},{"path":"/collections/{collection_id}/items","method":["GET"]},{"path":"/queryables","method":["GET"]},{"path":"/queryables/collections/{collection_id}/queryables","method":["GET"]},{"path":"/_mgmt/ping","method":["GET"]}],"dependencies":[{"method":"stac_fastapi.core.basic_auth.BasicAuth","kwargs":{"credentials":[{"username":"reader","password":"reader"}]}}]}]
     ports:
       - "8082:8082"
     volumes:

diff --git a/docker-compose.basic_auth_public.yml → docker-compose.route_dependencies_env.yml b/docker-compose.basic_auth_public.yml → docker-compose.route_dependencies_env.yml
@@ -22,7 +22,7 @@ services:
       - ES_USE_SSL=false
       - ES_VERIFY_CERTS=false
       - BACKEND=elasticsearch
-      - BASIC_AUTH={"public_endpoints":[{"path":"/","method":"GET"},{"path":"/conformance","method":"GET"},{"path":"/collections/{collection_id}/items/{item_id}","method":"GET"},{"path":"/search","method":"GET"},{"path":"/search","method":"POST"},{"path":"/collections","method":"GET"},{"path":"/collections/{collection_id}","method":"GET"},{"path":"/collections/{collection_id}/items","method":"GET"},{"path":"/queryables","method":"GET"},{"path":"/queryables/collections/{collection_id}/queryables","method":"GET"},{"path":"/_mgmt/ping","method":"GET"}],"users":[{"username":"admin","password":"admin","permissions":[{"path":"/","method":["GET"]},{"path":"/conformance","method":["GET"]},{"path":"/collections/{collection_id}/items/{item_id}","method":["GET","POST","PUT","DELETE"]},{"path":"/search","method":["GET","POST"]},{"path":"/collections","method":["GET","PUT","POST"]},{"path":"/collections/{collection_id}","method":["GET","DELETE"]},{"path":"/collections/{collection_id}/items","method":["GET","POST"]},{"path":"/queryables","method":["GET"]},{"path":"/queryables/collections/{collection_id}/queryables","method":["GET"]},{"path":"/_mgmt/ping","method":["GET"]}]}]}
+      - STAC_FASTAPI_ROUTE_DEPENDENCIES=[{"routes":[{"method":"GET","path":"/collections"}],"dependencies":[{"method":"conftest.must_be_bob"}]}]
     ports:
       - "8080:8080"
     volumes:
@@ -55,7 +55,7 @@ services:
       - ES_USE_SSL=false
       - ES_VERIFY_CERTS=false
       - BACKEND=opensearch
-      - BASIC_AUTH={"public_endpoints":[{"path":"/","method":"GET"},{"path":"/conformance","method":"GET"},{"path":"/collections/{collection_id}/items/{item_id}","method":"GET"},{"path":"/search","method":"GET"},{"path":"/search","method":"POST"},{"path":"/collections","method":"GET"},{"path":"/collections/{collection_id}","method":"GET"},{"path":"/collections/{collection_id}/items","method":"GET"},{"path":"/queryables","method":"GET"},{"path":"/queryables/collections/{collection_id}/queryables","method":"GET"},{"path":"/_mgmt/ping","method":"GET"}],"users":[{"username":"admin","password":"admin","permissions":[{"path":"/","method":["GET"]},{"path":"/conformance","method":["GET"]},{"path":"/collections/{collection_id}/items/{item_id}","method":["GET","POST","PUT","DELETE"]},{"path":"/search","method":["GET","POST"]},{"path":"/collections","method":["GET","PUT","POST"]},{"path":"/collections/{collection_id}","method":["GET","DELETE"]},{"path":"/collections/{collection_id}/items","method":["GET","POST"]},{"path":"/queryables","method":["GET"]},{"path":"/queryables/collections/{collection_id}/queryables","method":["GET"]},{"path":"/_mgmt/ping","method":["GET"]}]}]}
+      - STAC_FASTAPI_ROUTE_DEPENDENCIES=[{"routes":[{"method":"GET","path":"/collections"}],"dependencies":[{"method":"conftest.must_be_bob"}]}]
     ports:
       - "8082:8082"
     volumes:

diff --git a/docker-compose.route_dependencies_file.yml b/docker-compose.route_dependencies_file.yml
@@ -0,0 +1,129 @@
+version: '3.9'
+
+services:
+  app-elasticsearch:
+    container_name: stac-fastapi-es
+    image: stac-utils/stac-fastapi-es
+    restart: always
+    build:
+      context: .
+      dockerfile: dockerfiles/Dockerfile.dev.es
+    environment:
+      - STAC_FASTAPI_TITLE=stac-fastapi-elasticsearch
+      - STAC_FASTAPI_DESCRIPTION=A STAC FastAPI with an Elasticsearch backend
+      - STAC_FASTAPI_VERSION=3.0.0a1
+      - APP_HOST=0.0.0.0
+      - APP_PORT=8080
+      - RELOAD=true
+      - ENVIRONMENT=local
+      - WEB_CONCURRENCY=10
+      - ES_HOST=elasticsearch
+      - ES_PORT=9200
+      - ES_USE_SSL=false
+      - ES_VERIFY_CERTS=false
+      - BACKEND=elasticsearch
+      - STAC_FASTAPI_ROUTE_DEPENDENCIES=/app/route_dependencies/route_dependencies.json
+    ports:
+      - "8080:8080"
+    volumes:
+      - ./stac_fastapi:/app/stac_fastapi
+      - ./examples/route_dependencies:/app/route_dependencies
+      - ./scripts:/app/scripts
+      - ./esdata:/usr/share/elasticsearch/data
+    depends_on:
+      - elasticsearch
+    command:
+      bash -c "./scripts/wait-for-it-es.sh es-container:9200 && python -m stac_fastapi.elasticsearch.app"
+
+  app-opensearch:
+    container_name: stac-fastapi-os
+    image: stac-utils/stac-fastapi-os
+    restart: always
+    build:
+      context: .
+      dockerfile: dockerfiles/Dockerfile.dev.os
+    environment:
+      - STAC_FASTAPI_TITLE=stac-fastapi-opensearch
+      - STAC_FASTAPI_DESCRIPTION=A STAC FastAPI with an Opensearch backend
+      - STAC_FASTAPI_VERSION=2.1
+      - APP_HOST=0.0.0.0
+      - APP_PORT=8082
+      - RELOAD=true
+      - ENVIRONMENT=local
+      - WEB_CONCURRENCY=10
+      - ES_HOST=opensearch
+      - ES_PORT=9202
+      - ES_USE_SSL=false
+      - ES_VERIFY_CERTS=false
+      - BACKEND=opensearch
+      - STAC_FASTAPI_ROUTE_DEPENDENCIES=/app/route_dependencies/route_dependencies.json
+    ports:
+      - "8082:8082"
+    volumes:
+      - ./stac_fastapi:/app/stac_fastapi
+      - ./scripts:/app/scripts
+      - ./osdata:/usr/share/opensearch/data
+    depends_on:
+      - opensearch
+    command:
+      bash -c "./scripts/wait-for-it-es.sh os-container:9202 && python -m stac_fastapi.opensearch.app"
+
+  elasticsearch:
+    container_name: es-container
+    image: docker.elastic.co/elasticsearch/elasticsearch:${ELASTICSEARCH_VERSION:-8.11.0}
+    hostname: elasticsearch
+    environment:
+      ES_JAVA_OPTS: -Xms512m -Xmx1g
+    volumes:
+      - ./elasticsearch/config/elasticsearch.yml:/usr/share/elasticsearch/config/elasticsearch.yml
+      - ./elasticsearch/snapshots:/usr/share/elasticsearch/snapshots
+    ports:
+      - "9200:9200"
+
+  opensearch:
+    container_name: os-container
+    image: opensearchproject/opensearch:${OPENSEARCH_VERSION:-2.11.1}
+    hostname: opensearch
+    environment:
+      - discovery.type=single-node
+      - plugins.security.disabled=true
+      - OPENSEARCH_JAVA_OPTS=-Xms512m -Xmx512m
+    volumes:
+      - ./opensearch/config/opensearch.yml:/usr/share/opensearch/config/opensearch.yml
+      - ./opensearch/snapshots:/usr/share/opensearch/snapshots
+    ports:
+      - "9202:9202"
+
+  postgres:
+    image: postgres:15
+    container_name: postgres
+    hostname: keycloakdb
+    environment:
+      - POSTGRES_DB=keycloak
+      - POSTGRES_USER=keycloak
+      - POSTGRES_PASSWORD=password
+    volumes:
+      - postgres:/var/lib/postgresql/data
+
+  keycloak:
+    image: quay.io/keycloak/keycloak:25.0.0
+    container_name: keycloak
+    ports:
+      - 8083:8083
+    environment:
+      - KEYCLOAK_IMPORT=/tmp/keycloak-realm.json
+      - KEYCLOAK_ADMIN=admin
+      - KEYCLOAK_ADMIN_PASSWORD=admin
+      - KC_HTTP_PORT=8083
+      - KC_DB=postgres
+      - KC_DB_URL=jdbc:postgresql://keycloakdb:5432/keycloak
+      - KC_DB_USERNAME=keycloak
+      - KC_DB_PASSWORD=password
+    volumes:
+      - ./example/route_dependencies/stac-realm.json:/opt/keycloak/data/import
+    command: start-dev --import-realm
+    depends_on:
+      - postgres
+
+volumes:
+  postgres: