Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Update dbtutils tests #30

Merged
merged 2 commits into from
Mar 19, 2024
Merged

Update dbtutils tests #30

merged 2 commits into from
Mar 19, 2024

Conversation

sydneynotthecity
Copy link
Contributor

The dbtutils library introduced breaking changes during the major version upgrade from 0.9.5 to 1.1.1.

The recency test changed the expected data type from DATETIME to TIMESTAMP. This PR updates all dbtutils recency tests to reflect this change.

Updating these tests to closed_at, which is not the partition field on some tables, does not impact query performance or cost because these tests aggregate the field referenced:

select max(<timestamp>)
from `crypto-stellar.crypto_stellar.<table>`
..
..
..

In most cases, this query processes ~5GB or less, even for very large tables, like trust_lines.

@sydneynotthecity sydneynotthecity merged commit 3791096 into master Mar 19, 2024
1 check passed
@sydneynotthecity sydneynotthecity deleted the update-dbt-utils branch March 19, 2024 19:12
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

2 participants