You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
Hello everybody, please help with the following situation:
I have an AirByte connection already running for 72 hours trying to export a 277GB .tsv data set from an S3 Bucket into PostGress. All, the S3 Bucket, AirByte, and Postgress are installed in the same AWS instance which is powerful enough to process that data including 8 Virtual Processors and 1TB of disk space. The Airbyte connection is set to work with a block size of 10.000, if we try with a lower number the exporting process fails if we try with a higher number the process also fails. On top of that, after 72 hours and having exported 166GB the process failed and Airbyte initiated a new exporting attempt starting from 0. Is it normal that it takes so long to move the data from source to destination even though they al are in the same cloud instance?
reacted with thumbs up emoji reacted with thumbs down emoji reacted with laugh emoji reacted with hooray emoji reacted with confused emoji reacted with heart emoji reacted with rocket emoji reacted with eyes emoji
-
Hello everybody, please help with the following situation:
I have an AirByte connection already running for 72 hours trying to export a 277GB .tsv data set from an S3 Bucket into PostGress. All, the S3 Bucket, AirByte, and Postgress are installed in the same AWS instance which is powerful enough to process that data including 8 Virtual Processors and 1TB of disk space. The Airbyte connection is set to work with a block size of 10.000, if we try with a lower number the exporting process fails if we try with a higher number the process also fails. On top of that, after 72 hours and having exported 166GB the process failed and Airbyte initiated a new exporting attempt starting from 0. Is it normal that it takes so long to move the data from source to destination even though they al are in the same cloud instance?
Beta Was this translation helpful? Give feedback.
All reactions