Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Factor out more dbt YAML selectors #446

Open
jeancochrane opened this issue May 13, 2024 · 1 comment
Open

Factor out more dbt YAML selectors #446

jeancochrane opened this issue May 13, 2024 · 1 comment
Assignees
Labels
dbt Related to dbt (tests, docs, schema, etc)

Comments

@jeancochrane
Copy link
Contributor

In #373 we introduced a YAML selector in order to factor out a common node selection pattern that we use to run QC tests. We should factor out more of the node selection logic that we use in GitHub workflows so that everything is easier to reuse.

Note that since --selector overrides any behavior specified by other selection flags (issue context), we'll need to make use of YAML anchors and aliases in selectors.yml to make this code DRY.

Here are the patterns I think it would make sense to factor out:

  • Select modified or new nodes only (source)
    • Factor out --select state:modified state:new into a selector called new_or_modified and assign it a YAML anchor with the same name
  • Exclude QC tests (source)
    • Factor out --exclude tag:test_qc* into a new selector called exclude_qc_tests and assign it a YAML anchor with the same name
  • Select modified or new nodes only, excluding QC tests (source)
    • Union new_or_modified and exclude_qc_tests as defined above
  • Daily models (source 1 and source 2)
  • Factor out --select tag:daily into a new selector called daily
@dfsnow
Copy link
Member

dfsnow commented May 14, 2024

@wagnerlmichael This one goes to you and I'll review/supervise.

@dfsnow dfsnow assigned wagnerlmichael and unassigned Damonamajor May 14, 2024
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
dbt Related to dbt (tests, docs, schema, etc)
Projects
None yet
Development

No branches or pull requests

4 participants