Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Initial attempt at cleansing via sqlrender #241

Open
wants to merge 12 commits into
base: develop
Choose a base branch
from
Open

Initial attempt at cleansing via sqlrender #241

wants to merge 12 commits into from

Commits on Aug 14, 2021

  1. Initial attempt at cleansing via sqlrender

    @AnthonyMolinaro
    AnthonyMolinaro committed Aug 14, 2021
    Configuration menu
    Copy the full SHA
    1519615 View commit details
    Browse the repository at this point in the history

Commits on Aug 31, 2021

  1. cleansing plausble values - only updating, not deleting for plausible… 

    … hi/lo values
    @AnthonyMolinaro
    AnthonyMolinaro committed Aug 31, 2021
    Configuration menu
    Copy the full SHA
    34de580 View commit details
    Browse the repository at this point in the history

Commits on Sep 2, 2021

  1. continued cleansing work

    @AnthonyMolinaro
    AnthonyMolinaro committed Sep 2, 2021
    Configuration menu
    Copy the full SHA
    535564c View commit details
    Browse the repository at this point in the history

Commits on Sep 3, 2021

  1. continued cleansing work

    @AnthonyMolinaro
    AnthonyMolinaro committed Sep 3, 2021
    Configuration menu
    Copy the full SHA
    a85f8cc View commit details
    Browse the repository at this point in the history

Commits on Sep 9, 2021

  1. Initial updates to R code to incorporate optional cleansing

    @AnthonyMolinaro
    AnthonyMolinaro committed Sep 9, 2021
    Configuration menu
    Copy the full SHA
    007cb3a View commit details
    Browse the repository at this point in the history

Commits on Sep 10, 2021

  1. archive tables successfully created, but cleanse not yet working

    @AnthonyMolinaro
    AnthonyMolinaro committed Sep 10, 2021
    Configuration menu
    Copy the full SHA
    a403968 View commit details
    Browse the repository at this point in the history

  2. writing cleansing dml to files to serarate them from queries used in … 

    …execute phase
    @AnthonyMolinaro
    AnthonyMolinaro committed Sep 10, 2021
    Configuration menu
    Copy the full SHA
    ffd2d08 View commit details
    Browse the repository at this point in the history

Commits on Sep 13, 2021

  1. added logging info for cleansing step, excluding cdmField check from … 

    …cleanse
    @AnthonyMolinaro
    AnthonyMolinaro committed Sep 13, 2021
    Configuration menu
    Copy the full SHA
    4f6b0ba View commit details
    Browse the repository at this point in the history

  2. apparently, correlated subqueries using DELETE with EXISTS/NOT EXISTS… 

    … doesn't work on redshift. Switching to using INs
    @AnthonyMolinaro
    AnthonyMolinaro committed Sep 13, 2021
    Configuration menu
    Copy the full SHA
    d476f04 View commit details
    Browse the repository at this point in the history

Commits on Sep 17, 2021

  1. Truncating archive tables makes no sense, added addiional column to a… 

    …rchive tables to track when the archive occurred, minor fixes. Still need to add a try/catch so an error doesn't stop dqd
    @AnthonyMolinaro
    AnthonyMolinaro committed Sep 17, 2021
    Configuration menu
    Copy the full SHA
    3586b2f View commit details
    Browse the repository at this point in the history

Commits on Sep 20, 2021

  1. Added try/catch, moved post-prep cleanse execution into a function ra… 

    …ther than floating free code
    @AnthonyMolinaro
    AnthonyMolinaro committed Sep 20, 2021
    Configuration menu
    Copy the full SHA
    4c9a769 View commit details
    Browse the repository at this point in the history

Commits on Sep 22, 2021

  1. Added CAPTURE option, verified functionality, reenabled sqlOnly optio… 

    …n of EXECUTE
    @AnthonyMolinaro
    AnthonyMolinaro committed Sep 22, 2021
    Configuration menu
    Copy the full SHA
    062194e View commit details
    Browse the repository at this point in the history