[HUBBLE 444] Refactor Elementary monitoring to run every 30 min #413
Conversation
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
chowbao
reviewed
Jun 27, 2024
| enriched_history_operations_task = dbt_task(dag, tag="enriched_history_operations") | ||
| enriched_history_operations_task = dbt_task( | ||
| dag, tag="enriched_history_operations", excluded="singular_test" | ||
| ) | ||
| current_state_task = dbt_task(dag, tag="current_state") |
There was a problem hiding this comment.
Does this also need excluded="singular_test"?
There was a problem hiding this comment.
There's no need. I checked with the analytics team and this excluded tag for the singular test is related only to EHO, and the exclusion is mainly to avoid testing the same thing in two different workflows in a similar cadence.
chowbao
reviewed
Jun 27, 2024
| @@ -55,8 +54,6 @@ | |||
| snapshot_state = dbt_task(dag, tag="snapshot_state") | |||
| relevant_asset_trades = dbt_task(dag, tag="relevant_asset_trades") | |||
There was a problem hiding this comment.
Same question for the dbt_tasks in here
Do they need excluded="singular_test" as well?
There was a problem hiding this comment.
Same response to the first comment, there's no need.
chowbao
approved these changes
Jun 28, 2024
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
PR Checklist
PR Structure
otherwise).
Thoroughness
What
The
elementary_slack_alert_dbt_sdf_martstask was moved into a separate DAG that executes independently of the dbt models after some data quality tests, as defined in Alerting for dbt . This way, we will detect and trigger alerts on problems in a timely manner. The DAG is scheduled to run every 30 minutes, which aligns with thedbt_enriched_base_tablesDAG.The dbt tests running in the new DAG use a new tag designed to fit all the unit tests related to the models' data quality.
Why
Elementary monitoring and alerting are currently executed at the very end of the dbt DAGs. If an upstream dbt model fails, this means that elementary never alerts to a data quality issue because the alerting depends on DAG execution status. The result is that alerts are late arriving or arrive after an issue has already been resolved.
Known limitations
NA