Skip to content

Commit

Permalink
feat: Use FastCDC algorithm for chunking (#29)
Browse files Browse the repository at this point in the history
To minimize the number of chunks that need to be resent from server to client use content defined chunking.  

Uses a FastCDC implementation based on https://github.com/ronomon/deduplication/blob/master/binding.js

Currently uses an average chunk size of 1KiB, as this is the size of value we recommend to users and perf test.   In practice this achieves a very high percentage of deduplication across document versions (typically 97+% for typing in documents > 100KiB).

Updates the provider to use experimentalWatch instead of subscribe to avoid the need to deserialize unchanged chunks (from base64 strings to Uint8Array).

Also optimizes the client side mutator implementation to just write the client update value and not merge with the server update.  We only apply the client update on initial load of a document as all other client updates will have originated from that ydoc instance and so do not need to be applied.
  • Loading branch information
grgbkr authored Dec 13, 2023
1 parent 57dcfb1 commit 1c00214
Show file tree
Hide file tree
Showing 7 changed files with 11,758 additions and 69 deletions.
Loading

0 comments on commit 1c00214

Please sign in to comment.