diff --git a/content/en/hosting/analytics/building-dbt-models.md b/content/en/hosting/analytics/building-dbt-models.md index ba5400cb7..00f5c9737 100644 --- a/content/en/hosting/analytics/building-dbt-models.md +++ b/content/en/hosting/analytics/building-dbt-models.md @@ -37,7 +37,10 @@ packages: ``` To avoid breaking changes in downstream models, include `revision` in the dependency, which should be a version tag for `cht-pipeline`. -In the CHT Sync config, set the URL of dbt GitHub repository to the `CHT_PIPELINE_BRANCH_URL` [environment variable]({{< relref "hosting/analytics/environment-variables" >}}), either in `.env` if using Docker compose, or in `values.yaml` if using Kubernetes. +It is helpful to test changes locally before commiting them to a remote repository. +For local testing and devleopment, in the CHT Sync config, set the path to the dbt project to the `DBT_LOCAL_PATH` [environment variable]({{< relref "hosting/analytics/environment-variables" >}}) in `.env` +Use the [docker compose setup]({{< relref "hosting/analytics/setup-docker-compose" >}}) for CHT Sync and use `docker-compose.local.yml` instead of `docker-compose.yml`. +For production and remote deployments, in the CHT Sync config, set the URL of dbt GitHub repository to the `CHT_PIPELINE_BRANCH_URL` [environment variable]({{< relref "hosting/analytics/environment-variables" >}}), either in `.env` if using Docker compose, or in `values.yaml` if using Kubernetes. {{% alert title="Note" %}} If `CHT_PIPELINE_BRANCH_URL` is pointing to a private GitHub repository, you'll need an access token in the URL. Assuming your repository is `medic/cht-pipeline`, you would replace `` with an access token: `https://@github.com/medic/cht-pipeline.git#main`. Please see [GitHub's instructions](https://docs.github.com/en/authentication/keeping-your-account-and-data-secure/managing-your-personal-access-tokens) on how to generate a token. If you create a fine-grained access token you need to provide read and write access to the [contents](https://docs.github.com/en/rest/authentication/permissions-required-for-fine-grained-personal-access-tokens?apiVersion=2022-11-28#repository-permissions-for-contents) of the repository. diff --git a/content/en/hosting/analytics/environment-variables.md b/content/en/hosting/analytics/environment-variables.md index 33960c6e1..2887d7806 100644 --- a/content/en/hosting/analytics/environment-variables.md +++ b/content/en/hosting/analytics/environment-variables.md @@ -24,13 +24,14 @@ All the variables in the `.env` file: | `POSTGRES_SCHEMA` | `v1` | PostgreSQL schema | | `POSTGRES_TABLE` | `couchdb` | PostgreSQL table where the CouchDB data is copied | | `POSTGRES_HOST` | `postgres` | PostgreSQL instance | -| `POSTGRES_PORT` | `5432` | PostgreSQL port | -| `CHT_PIPELINE_BRANCH_URL` | `"https://github.com/medic/cht-pipeline.git#main"` | cht-pipeline branch containing the dbt models | +| `POSTGRES_PORT` | `5432` | PostgreSQL port | +| `CHT_PIPELINE_BRANCH_URL` | `"https://github.com/medic/cht-pipeline.git#main"` | cht-pipeline branch containing the dbt models | +| `DBT_LOCAL_PATH` | `""` | if running dbt locally, path to the dbt project on the host | | `DATAEMON_INTERVAL` | `5` | Interval (in minutes) for looking for new changes in the CouchDB data | -| `COUCHDB_USER` | `medic` | Username of the CouchDB instance | -| `COUCHDB_PASSWORD` | `password` | Password of the CouchDB instance | -| `COUCHDB_DBS` | `"medic"` | Comma separated list of databases to sync e.g `"medic, medic_sentinel"` | -| `COUCHDB_HOST` | `couchdb` | Host of the CouchDB instance | -| `COUCHDB_PORT` | `5984` | Port of the CouchDB instance | -| `COUCHDB_SECURE` | `false` | Is connection to CouchDB instance secure? | +| `COUCHDB_USER` | `medic` | Username of the CouchDB instance | +| `COUCHDB_PASSWORD` | `password` | Password of the CouchDB instance | +| `COUCHDB_DBS` | `"medic"` | Comma separated list of databases to sync e.g `"medic, medic_sentinel"` | +| `COUCHDB_HOST` | `couchdb` | Host of the CouchDB instance | +| `COUCHDB_PORT` | `5984` | Port of the CouchDB instance | +| `COUCHDB_SECURE` | `false` | Is connection to CouchDB instance use HTTPS? | diff --git a/content/en/hosting/analytics/setup-docker-compose.md b/content/en/hosting/analytics/setup-docker-compose.md index 98bf5e79c..43d530d01 100644 --- a/content/en/hosting/analytics/setup-docker-compose.md +++ b/content/en/hosting/analytics/setup-docker-compose.md @@ -30,6 +30,15 @@ For production CHT Core deployments, the port will most likely need to be set to The first time you run the commands from any of the sections below it will need to download many Docker images and will take a while. You'll know it's done when you see `#8 DONE 0.0s` and you are returned to the command line. Be patient! {{% /alert %}} +### Running DBT models locally + +When developing DBT models, it is helpful to test changes locally before commiting them to a remote repository. +Set the path to the project to the `DBT_LOCAL_PATH` [environment variable]({{< relref "hosting/analytics/environment-variables" >}}) in `.env` and use `docker-compose.local.yml`. + +```sh +docker compose -f docker-compose.postgres.yml -f docker-compose.local.yml up -d +``` + ### Separate CouchDB instance This setup involves starting couch2pg, PostgreSQL, pgAdmin and dbt. It assumes you have a CouchDB instance running, and you updated the `.env` CouchDB variables accordingly.