does the plugin uses s3 as an append-only database?

[fyears]

(I am the author of remotely-save)

To my surprise livesync adds support for s3 recently. So (obviously) out of curiosity I give it a try. :-)

It seems that lots of jsonl.gz files are generated in s3. So does live sync uses s3 as an append-only database, instead of raw file storage?

[berot3]

Any news?

[vrtmrz]

Wow! I am very honoured to receive your message! I would love to express great respect and appreciation to you and your codebase. It has so helped me! However, sorry for being quite late in replying!

It seems that lots of jsonl.gz files are generated in s3. So does live sync uses s3 as an append-only database, instead of raw file storage?

Yes, Self-hosted LiveSync uses an S3 bucket as an append-only storage, like to save journals.
The files sent and received do not represent files. It consisted of the metadata of the files or the differences that have occurred.

Self-hosted LiveSync splits the file into several chunks. These chunks have the id determined by its content. And, metadata keeps the list of chunk IDs instead of its content. So we only have to exchange missing chunks and journals.

Fortunately, here is a design document. I hope that this helps us.