You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
{{ message }}
This repository has been archived by the owner on Apr 11, 2024. It is now read-only.
Please describe the feature you'd like to see
There are multiple scenarios to handle dataset transfers:
Lets take a scenario where we are transferring 100 rows from snowflake to bigquery and error occurs after transferring the 99th row. What's the correct way to handle this?
By default should this be a TRANSACTION and roll back all the changes done?
Or should it throw the error for all the rows that error out? What's the right way to error out for a larger number of rows that has the error
This should be handled before 1.0 release. To start with we can replicate something similar to Fivetran does.
Acceptance Criteria
All checks and tests in the CI should pass
Unit tests (90% code coverage or more)
Integration tests (if the feature relates to a new database or external service)
Example DAG
Docstrings in reStructuredText for each of methods, classes, functions and module-level attributes (including Example DAG on how it should be used)
Exception handling in case of errors
Logging (are we exposing useful information to the user? e.g. source and destination)
Improve the documentation (README, Sphinx, and any other relevant)
Please describe the feature you'd like to see
There are multiple scenarios to handle dataset transfers:
This should be handled before 1.0 release. To start with we can replicate something similar to Fivetran does.
Acceptance Criteria
The text was updated successfully, but these errors were encountered: