[Request for Beta Testers] Destination S3 Data Lake is entering early access!

[edgao]

Early Access coming soon!

Hi Airbyte community,

Early access for destination-s3-data-lake will soon be available. This is the evolution of our existing Apache Iceberg destination, bringing you:

• Airbyte 1.0 features: destination-s3-data-lake now supports refreshes / resumable full refresh.
• Iceberg support: Iceberg is the first data lake format this connector can write, with support for the Glue and Nessie (new!) catalogs.
• Reliability improvements: We now leverage Iceberg's snapshot branching capabilities to checkpoint and recover from failed syncs.
• Strongly-typed tables: The new connector writes data with true Iceberg types, instead of a single JSON string.
• Sync mode support: Dedup mode (via merge-on-read) is now available.

Comment here if you'd like to be added to our private slack channel. We'll kick off after the holidays are over.

How to Enroll

We are actively seeking and iterating on feedback to refine these features prior to general availability. We'll release more information about how to start using the new connector in early January. At the moment, we're only making this available to OSS users. If you use Airbyte Cloud and want to participate in this early access, please comment below!

Request for Feedback

As you explore the new functionality, we ask that you pay special attention to table and column name changes, missing or incorrect column types, and the speed and cost of your syncs. We'll send out more details, including some specific questions, after we begin the beta.

We look forward to hearing about your experience with S3 data lakes on Apache Iceberg. Please don't hesitate to share any feedback or questions in the comments below. You can also join us on Slack if you would like to chat directly with the team about this feature.