Commit
This commit does not belong to any branch on this repository, and may belong to a fork outside of the repository.
Set cursors to 0 when streaming (#515)
We need to land xmtp/xmtp-node-go#351 in order to implement `streamAllMessages()` without gaps. However once we have actual cursor support on the server, we may introduce bugs in streaming of welcomes and messages within a single group, because the client doesn't have a consistent strategy for setting the cursor for streams yet. The easiest way to avoid regressions/bugs is to keep the existing behavior identical - we can do this by setting the cursor to 0, which effectively disables the sync from DB on the server. The existing streaming methods will just stream whatever is newly coming in from the Waku relay. ## What I think should be done in the long-term In a nutshell, we are trying to synchronize three views of the data - The node/backend database -> libxmtp database -> integrator's in-memory view of the data In order to avoid gaps when requesting new data, each layer needs to pass a cursor down to the next layer representing the data they currently have. We are already doing this for the pull-based methods, with the integrator's `FfiGroup.find_messages()` and libxmtp's `MlsGroup.sync()`, which both allow you to specify where you want to pull from. We want to do this for the push-based streaming methods too. 1) Allow the *integrator* to pass down the initial cursor they want to stream from. This forces the integrator to avoid gaps in their data. Libxmtp can then pass this cursor along to the node backend. If the cursor is *greater* than the last message in the libxmtp database, libxmtp can additionally call `sync()` first before starting the streaming in order to ensure libxmtp's DB has no gaps. 2) For situations where libxmtp wants to close/re-open streams under the hood, libxmtp should hold an in-memory cursor. Libxmtp should save the in-memory cursor for the stream being closed, then pass the same cursor to the new stream being opened. Feedback on this plan welcome!
- Loading branch information