Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Fix test errors in source tables #20

Merged
merged 2 commits into from
Feb 15, 2024
Merged

Fix test errors in source tables #20

Show file tree
Hide file tree
Changes from 1 commit
Commits
Show all changes
2 commits
Select commit Hold shift + click to select a range
361a192
Fix test errors in source tables
chowbao Feb 15, 2024
ea0d97d
fixes
chowbao Feb 15, 2024
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
2 changes: 1 addition & 1 deletion models/sources/src_config_settings.yml
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -9,7 +9,7 @@ sources:
tests:
- dbt_utils.recency:
datepart: hour
field: closed_at
field: cast(closed_at as datetime)
interval: 12
config:
severity: warn
Expand Down
2 changes: 1 addition & 1 deletion models/sources/src_contract_code.yml
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -9,7 +9,7 @@ sources:
tests:
- dbt_utils.recency:
datepart: hour
field: closed_at
field: cast(closed_at as datetime)
interval: 12
config:
severity: warn
Expand Down
15 changes: 1 addition & 14 deletions models/sources/src_contract_data.yml
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -9,26 +9,13 @@ sources:
tests:
- dbt_utils.recency:
datepart: hour
field: closed_at
field: cast(closed_at as datetime)
interval: 12
config:
severity: warn
meta:
description:
"Monitors the freshness of your table over time, as the expected time between data updates."
- dbt_utils.unique_combination_of_columns:
combination_of_columns:
- contract_id
- contract_key_type
- contract_durability
- last_modified_ledger
- ledger_entry_change
config:
where: closed_at >= datetime_trunc(datetime_sub(current_timestamp(), interval 2 day), day)
and closed_at < datetime_trunc(current_timestamp(), day)
meta:
description:
"Tests the uniqueness combination of: contract_id, contract_key_type, contract_durability, last_modified_ledger, and ledger_entry_change."
columns:
- name: contract_id
description: '{{ doc("contract_id") }}'
Expand Down
2 changes: 1 addition & 1 deletion models/sources/src_history_assets.yml
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -18,7 +18,7 @@ sources:
"Monitors the freshness of your table over time, as the expected time between data updates."
columns:
- name: assets_id
description: '{{ doc("asset_type") }}'
description: '{{ doc("assets_id") }}'
tests:
- not_null:
config:
Expand Down
2 changes: 1 addition & 1 deletion models/sources/src_history_operations.yml
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -9,7 +9,7 @@ sources:
tests:
- dbt_utils.recency:
datepart: hour
field: closed_at
field: cast(closed_at as datetime)
sydneynotthecity marked this conversation as resolved.
Show resolved Hide resolved
interval: 12
config:
severity: warn
Expand Down
46 changes: 23 additions & 23 deletions models/sources/src_history_transactions.yml
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -9,7 +9,7 @@ sources:
tests:
- dbt_utils.recency:
datepart: hour
field: closed_at
field: cast(closed_at as datetime)
interval: 12
config:
severity: warn
Expand All @@ -22,25 +22,25 @@ sources:
tests:
- unique:
config:
where: closed_at >= datetime_trunc(datetime_sub(current_datetime(), interval 2 day), day)
and closed_at < datetime_trunc(current_datetime(), day)
where: cast(closed_at as datetime) >= datetime_trunc(datetime_sub(current_datetime(), interval 2 day), day)
and cast(closed_at as datetime) < datetime_trunc(current_datetime(), day)
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

might need to add the partition fields to this where clause otherwise it's gonna be expensive

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Yeah I'll look at these tables again. It should be based on the partitioned field

- not_null:
config:
where: closed_at > current_datetime - interval 2 day
where: closed_at > current_timestamp - interval 2 day

- name: transaction_hash
description: '{{ doc("transaction_hash") }}'
tests:
- not_null:
config:
where: closed_at > current_datetime - interval 2 day
where: closed_at > current_timestamp - interval 2 day

- name: ledger_sequence
description: '{{ doc("ledger_sequence") }}'
tests:
- not_null:
config:
where: closed_at > current_datetime - interval 2 day
where: closed_at > current_timestamp - interval 2 day

- name: application_order
description: '{{ doc("application_order") }}'
Expand All @@ -50,42 +50,42 @@ sources:
tests:
- not_null:
config:
where: closed_at > current_datetime - interval 2 day
where: closed_at > current_timestamp - interval 2 day

- name: account_sequence
description: '{{ doc("account_sequence") }}'
tests:
- not_null:
config:
where: closed_at > current_datetime - interval 2 day
where: closed_at > current_timestamp - interval 2 day

- name: max_fee
description: '{{ doc("max_fee") }}'
tests:
- not_null:
config:
where: closed_at > current_datetime - interval 2 day
where: closed_at > current_timestamp - interval 2 day

- name: operation_count
description: '{{ doc("transaction_operation_count") }}'
tests:
- not_null:
config:
where: closed_at > current_datetime - interval 2 day
where: closed_at > current_timestamp - interval 2 day

- name: created_at
description: '{{ doc("created_at") }}'
tests:
- not_null:
config:
where: closed_at > current_datetime - interval 2 day
where: closed_at > current_timestamp - interval 2 day

- name: memo_type
description: '{{ doc("memo_type") }}'
tests:
- not_null:
config:
where: closed_at > current_datetime - interval 2 day
where: closed_at > current_timestamp - interval 2 day

- name: memo
description: '{{ doc("memo") }}'
Expand All @@ -95,21 +95,21 @@ sources:
tests:
- not_null:
config:
where: closed_at > current_datetime - interval 2 day
where: closed_at > current_timestamp - interval 2 day

- name: successful
description: '{{ doc("successful") }}'
tests:
- not_null:
config:
where: closed_at > current_datetime - interval 2 day
where: closed_at > current_timestamp - interval 2 day

- name: fee_charged
description: '{{ doc("fee_charged") }}'
tests:
- not_null:
config:
where: closed_at > current_datetime - interval 2 day
where: closed_at > current_timestamp - interval 2 day

- name: inner_transaction_hash
description: '{{ doc("inner_transaction_hash") }}'
Expand Down Expand Up @@ -146,56 +146,56 @@ sources:
tests:
- not_null:
config:
where: closed_at > current_datetime - interval 2 day
where: closed_at > current_timestamp - interval 2 day

- name: tx_result
description: '{{ doc("tx_result") }}'
tests:
- not_null:
config:
where: closed_at > current_datetime - interval 2 day
where: closed_at > current_timestamp - interval 2 day

- name: tx_meta
description: '{{ doc("tx_meta") }}'
tests:
- not_null:
config:
where: closed_at > current_datetime - interval 2 day
where: closed_at > current_timestamp - interval 2 day

- name: tx_fee_meta
description: '{{ doc("tx_fee_meta") }}'
tests:
- not_null:
config:
where: closed_at > current_datetime - interval 2 day
where: closed_at > current_timestamp - interval 2 day

- name: batch_id
description: '{{ doc("batch_id") }}'
tests:
- not_null:
config:
where: closed_at > current_datetime - interval 2 day
where: closed_at > current_timestamp - interval 2 day

- name: batch_run_date
description: '{{ doc("batch_run_date") }}'
tests:
- not_null:
config:
where: closed_at > current_datetime - interval 2 day
where: closed_at > current_timestamp - interval 2 day

- name: batch_insert_ts
description: '{{ doc("batch_insert_ts") }}'
tests:
- not_null:
config:
where: closed_at > current_datetime - interval 2 day
where: closed_at > current_timestamp - interval 2 day

- name: closed_at
description: '{{ doc("closed_at") }}'
tests:
- not_null:
config:
where: closed_at > current_datetime - interval 2 day
where: closed_at > current_timestamp - interval 2 day

- name: refundable_fee
description: '{{ doc("refundable_fee") }}'
Expand Down
24 changes: 12 additions & 12 deletions models/sources/src_ttl.yml
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -9,7 +9,7 @@ sources:
tests:
- dbt_utils.recency:
datepart: hour
field: closed_at
field: cast(closed_at as datetime)
interval: 12
config:
severity: warn
Expand All @@ -23,8 +23,8 @@ sources:
- last_modified_ledger
- ledger_entry_change
config:
where: closed_at >= datetime_trunc(datetime_sub(current_datetime(), interval 2 day), day)
and closed_at < datetime_trunc(current_datetime(), day)
where: cast(closed_at as datetime) >= datetime_trunc(datetime_sub(current_datetime(), interval 2 day), day)
and cast(closed_at as datetime) < datetime_trunc(current_datetime(), day)
meta:
description:
"Tests the uniqueness combination of: key_hash, live_until_ledger_seq, last_modified_ledger, and ledger_entry_change."
Expand All @@ -34,60 +34,60 @@ sources:
tests:
- not_null:
config:
where: closed_at > current_datetime - interval 2 day
where: closed_at > current_timestamp - interval 2 day

- name: live_until_ledger_seq
description: '{{ doc("live_until_ledger_seq") }}'
tests:
- not_null:
config:
where: closed_at > current_datetime - interval 2 day
where: closed_at > current_timestamp - interval 2 day

- name: last_modified_ledger
description: '{{ doc("last_modified_ledger") }}'
tests:
- not_null:
config:
where: closed_at > current_datetime - interval 2 day
where: closed_at > current_timestamp - interval 2 day

- name: ledger_entry_change
description: '{{ doc("ledger_entry_change") }}'
tests:
- not_null:
config:
where: closed_at > current_datetime - interval 2 day
where: closed_at > current_timestamp - interval 2 day

- name: deleted
description: '{{ doc("deleted") }}'
tests:
- not_null:
config:
where: closed_at > current_datetime - interval 2 day
where: closed_at > current_timestamp - interval 2 day

- name: batch_id
description: '{{ doc("batch_id") }}'
tests:
- not_null:
config:
where: closed_at > current_datetime - interval 2 day
where: closed_at > current_timestamp - interval 2 day

- name: batch_run_date
description: '{{ doc("batch_run_date") }}'
tests:
- not_null:
config:
where: closed_at > current_datetime - interval 2 day
where: closed_at > current_timestamp - interval 2 day

- name: batch_insert_ts
description: '{{ doc("batch_insert_ts") }}'
tests:
- not_null:
config:
where: closed_at > current_datetime - interval 2 day
where: closed_at > current_timestamp - interval 2 day

- name: closed_at
description: '{{ doc("closed_at") }}'
tests:
- not_null:
config:
where: closed_at > current_datetime - interval 2 day
where: closed_at > current_timestamp - interval 2 day