You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
We are setting up AirByte to perform “full refresh/overwrite” on each table in our configuration due to issues getting CDC to work reliably. The goal is to minimize the time that these refreshed tables in the destination have no data. In the past when “full refresh/overwrite” is run, it clears out the destination table and then inserts the data back in. During the loading phase, tables are without data leading to downstream errors.
I have two questions:
What is the best practice to perform a full reset that best minimizes downtime?
Is there a way to full refresh these tables one-by-one in order to limit the unavailability of the source data while it is being written to the destination?
reacted with thumbs up emoji reacted with thumbs down emoji reacted with laugh emoji reacted with hooray emoji reacted with confused emoji reacted with heart emoji reacted with rocket emoji reacted with eyes emoji
-
We are setting up AirByte to perform “full refresh/overwrite” on each table in our configuration due to issues getting CDC to work reliably. The goal is to minimize the time that these refreshed tables in the destination have no data. In the past when “full refresh/overwrite” is run, it clears out the destination table and then inserts the data back in. During the loading phase, tables are without data leading to downstream errors.
I have two questions:
What is the best practice to perform a full reset that best minimizes downtime?
Is there a way to full refresh these tables one-by-one in order to limit the unavailability of the source data while it is being written to the destination?
Thanks in advance
Beta Was this translation helpful? Give feedback.
All reactions