Skip to content

Commit

Permalink
Merge pull request #266 from dbt-labs/feature/display-results
Browse files Browse the repository at this point in the history
Add macros to print issues to logs
  • Loading branch information
dave-connors-3 authored Jan 26, 2023
2 parents 40755ee + 53fe3a4 commit a955400
Show file tree
Hide file tree
Showing 3 changed files with 77 additions and 2 deletions.
22 changes: 20 additions & 2 deletions README.md
Original file line number Diff line number Diff line change
Expand Up @@ -47,7 +47,7 @@ Once you've installed the package, all you have to do is run a `dbt build --sele

Each test warning indicates the presence of a type of misalignment. To troubleshoot a misalignment:
1. Locate the related documentation below
2. Query the associated model to find the specific instances of the issue within your project
2. Query the associated model to find the specific instances of the issue within your project or set up an [`on-run-end` hook](https://docs.getdbt.com/reference/project-configs/on-run-start-on-run-end) to display the rules violations in the dbt logs (see [displaying violations in the logs](#displaying-violations-in-the-logs))
3. Either fix the issue(s) or [customize](#customization) the package to exclude them

----
Expand Down Expand Up @@ -1123,10 +1123,28 @@ seeds:
#### 3. Run the seed and the package

We then run both the seed and the package by executing the following command:
```

```bash
dbt build --select package:dbt_project_evaluator dbt_project_evaluator_exceptions
```

### Displaying violations in the logs

This package provides a macro that can be executed via an `on-run-end` hook to display the package results in the logs in addition to storing those in the Data Warehouse.

To use it, you can add the following line in your `dbt_project.yml`:

```yml
on-run-end: "{{ dbt_project_evaluator.print_dbt_project_evaluator_issues() }}"
```

In the case that you are storing the tables with the package results in a schema or database different from the default ones from your profile, the following parameters are available for `print_dbt_project_evaluator_issues()`:

- `schema_project_evaluator`: the schema where the tables are stored
- `db_project_evaluator`: the database where the tables are stored

# dbt_project.yml

----
## Running this package as a CI check

Expand Down
2 changes: 2 additions & 0 deletions integration_tests_2/dbt_project.yml
Original file line number Diff line number Diff line change
Expand Up @@ -28,6 +28,8 @@ dispatch:
- macro_namespace: dbt
search_order: ['dbt_project_evaluator', 'dbt']

on-run-end: "{{ dbt_project_evaluator.print_dbt_project_evaluator_issues() }}"

models:
dbt_project_evaluator_integration_tests_2:
# materialize as ephemeral to prevent the fake models from executing, but keep them enabled
Expand Down
55 changes: 55 additions & 0 deletions macros/on-run-end/print_dbt_project_evaluator_issues.sql
Original file line number Diff line number Diff line change
@@ -0,0 +1,55 @@
{% macro _return_list_header_rows(query) %}

{%- call statement('get_query_results', fetch_result=True,auto_begin=false) -%}

{{ query }}

{%- endcall -%}

{% set sql_results=[] %}

{%- if execute -%}
{% set sql_results_table = load_result('get_query_results').table %}
{% do sql_results.append(sql_results_table.column_names) %}
{% for row_data in sql_results_table.rows %}
{% do sql_results.append(row_data.values()) %}
{% endfor %}
{%- endif -%}

{{ return(sql_results) }}

{% endmacro %}


{% macro print_dbt_project_evaluator_issues(schema_project_evaluator=None, db_project_evaluator=None) %}

{%- if flags.WHICH in ["build","test"] -%}
{{ print("\n### List of issues raised by dbt_project_evaluator ###") }}

{% for result in results | selectattr('failures') | selectattr('failures', '>', 0) %}

{% if result.node.fqn[0] == "dbt_project_evaluator" %}

{{ print("\n-- " ~ result.node.alias ~ " --") }}

{% set model_checked = result.node.depends_on.nodes[0].split('.')[-1] %}
{% set db_schema = database_package ~ "." ~ schema_package if database_package else schema_package %}
{% set db_schema_model = db_schema ~ "." ~ model_checked if db_schema else model_checked %}

{% set sql_statement %}
select * from {{ db_schema_model }}
{% endset %}

{%- set failures = dbt_project_evaluator._return_list_header_rows(sql_statement) -%}
{% for row in failures %}
{{ print(row | join(", ")) }}
{% endfor %}

{% endif %}

{% endfor %}

{{ print("\n") }}
{%- endif %}

{% endmacro %}

0 comments on commit a955400

Please sign in to comment.