-
Notifications
You must be signed in to change notification settings - Fork 4
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Update soroban pipelines #17
Conversation
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Looks good, just had some questions for my own understanding of the data model
description: '{{ doc("ttl_current") }}' | ||
tests: | ||
- dbt_utils.recency: | ||
datepart: hour |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
does this truncate the timestamp to the hourly level? Like if data was older than 24+ hours, would we get an alert? or is this truly evaluating only the hour part of the datetime field?
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
The truncate should be fine because datetime still has the hh:mm:ss part of the timestamp. I'm not sure about the 24+ hours alert. I would hope that dbt_utils is smart enough to take the datetime_diff instead of the extract(hour) - extract(hour)
We should probably test it but also we would get an error before this would be a problem because the _current table are run every 30 mins so we would hit the 12 hour recency test regardless
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
sounds good. let's test and adjust appropriately
Add Soroban pipelines and changes
stg_history_operations
,stg_history_effects
, andstg_history_transactions
with new soroban columns