Skip to content

Commit

Permalink
Merge pull request #105 from OHDSI/86-docs-improve-customise-default-…
Browse files Browse the repository at this point in the history
…overview-page-of-dbt-docs

docs: add `overview` documentation
  • Loading branch information
lawrenceadams authored Nov 29, 2024
2 parents d6fe36d + e42d871 commit 2ea5a6d
Show file tree
Hide file tree
Showing 4 changed files with 51 additions and 0 deletions.
2 changes: 2 additions & 0 deletions .github/workflows/generate-docs.yml
Original file line number Diff line number Diff line change
Expand Up @@ -44,10 +44,12 @@ jobs:
- name: Generate Docs and prepare for upload
run: |
dbt docs generate
sed -i "s#<img style=\"width: 100px; height: 40px\" class=\"logo\" ng-src=\"{{ logo }}\" />#<img style=\"width: 140px\" src=\"assets/OHDSI-logo-with-text-horizontal-colored-white-background.png\" />#g" ./target/index.html
mkdir ./payload
mv ./target/index.html ./payload
mv ./target/manifest.json ./payload
mv ./target/catalog.json ./payload
mv ./target/assets ./payload
- name: Upload artifact
uses: actions/upload-pages-artifact@v3
Expand Down
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
2 changes: 2 additions & 0 deletions dbt_project.yml
Original file line number Diff line number Diff line change
Expand Up @@ -4,6 +4,8 @@ config-version: 2

profile: 'synthea_omop_etl'

asset-paths: ["assets"]
docs-paths: ["docs"]
model-paths: ["models"]
analysis-paths: ["analyses"]
test-paths: ["tests"]
Expand Down
47 changes: 47 additions & 0 deletions docs/overview.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,47 @@
{% docs __overview__ %}

## dbt-synthea

dbt-synthea is a project to demonstrate the power of dbt for building OMOP ETLs. Using [Synthea](https://synthetichealth.github.io/synthea/), it is possible to generate a synthetic patient dataset which can be transformed into the OMOP standard.

### Project Structure

dbt uses parameterized SQL files called [“models”](https://docs.getdbt.com/docs/build/models), which are run in a correct order by building a Directed Acyclic Graph (DAG).

The models have been setup with three distinct stages:
- **Staging**: SQL queries that are 1:1 with the source Synthea and vocabulary tables and are used to adjust data types and column names as needed
- **Intermediate**: SQL queries used to perform the bulk of the joining and transformation needed to move from source tables into the OMOP CDM, with a focus on joins and transformations which may be reused several times in the final models; for example, source-to-standard concept mappings or modeling of different visit types.
- **Mart**: SQL queries that are 1:1 with the final OMOP tables.

The data originates from either [seed files](https://github.com/OHDSI/dbt-synthea/tree/main/seeds) or with data loaded into a database system of choice: the [developer setup instructions](https://github.com/OHDSI/dbt-synthea?tab=readme-ov-file#developer-setup) provide more information.

## Navigation
You can use the Project and Database navigation tabs on the left side of the window to explore the models in your project.

### Project Tab
The Project tab mirrors the directory structure of your dbt project. In this tab, you can see all of the models defined in your dbt project, as well as models imported from dbt packages.

### Database Tab
The Database tab also exposes your models, but in a format that looks more like a database explorer. This view shows relations (tables and views) grouped into database schemas. Note that ephemeral models are not shown in this interface, as they do not exist in the database.

### Graph Exploration
You can click the blue icon on the bottom-right corner of the page to view the lineage graph of your models.

On model pages, you'll see the immediate parents and children of the model you're exploring. By clicking the Expand button at the top-right of this lineage pane, you'll be able to see all of the models that are used to build, or are built from, the model you're exploring.

Once expanded, you'll be able to use the --select and --exclude model selection syntax to filter the models in the graph. For more information on model selection, check out the dbt docs.

Note that you can also right-click on models to interactively filter and explore the graph.

## More Information

- [What is dbt](https://docs.getdbt.com/docs/introduction)?
- [What is OMOP](https://www.ohdsi.org/data-standardization/)?
- [What is Synthea](https://synthetichealth.github.io/synthea/)?
- [Join OHDSI and collaborate](https://www.ohdsi.org/join-the-journey/)
- [Installation](https://github.com/OHDSI/dbt-synthea?tab=readme-ov-file#developer-setup)
- [Project Abstract](https://www.ohdsi.org/wp-content/uploads/2024/10/124-Sadowski-dbt-synthea-Abstract-Julien-Nakache.pdf)

![OMOP Logo](assets/OHDSI-logo-with-text-horizontal-colored-white-background.png)

{% enddocs %}

0 comments on commit 2ea5a6d

Please sign in to comment.