gRPC subscribers should be able to detect missing updates

[grooviegermanikus]

Requirement

As gRPC subscriber I want to be able to detect if I have missed an account update message.

Proposed solution

Let's assume a contiguous stream of updates (e.g. account updates).

Per subscriber the updates sent should get assigned an "index" value to MessageAccount which is monotonic/incrementing starting from "0"

Comment on write_version: this version is global (to all accounts notified); it is useful to know which update is more recent but not to detect missing updates.

Open Questions

• Q: Should we allow multiple messages with same index or should plugin prevent that?

[vovkman]

Wouldn't you detect this by seeing the stream disconnected? It seems that all error scenarios disconnect the stream

[linuskendall]

Wouldn't you detect this by seeing the stream disconnected? It seems that all error scenarios disconnect the stream

Does gRPC guarantee no messages lost? I'm not clear about this whether this is actually a guarantee of the protocol or not? IIRC gRPC does do in-order delivery in bidi strems which seems to be what this page also claim:

The order of messages in each stream is preserved.`:
https://grpc.io/docs/what-is-grpc/core-concepts/#:~:text=Again%20gRPC%20guarantees%20message%20ordering,using%20a%20read%2Dwrite%20stream.

This would mean that unless the Rust gRPC client can drop messages on the client end (or re-order) we would already be guaranteed that we receive each message into our client software in order? If missing messages (i.e received N, N+2 but not N+1) causes stream disconnect then we are good.

[grooviegermanikus]

I'm a bit paranoid on that.

Sure, it would be useful if the underlaying stream can give these guarantees yet it would be useful if the layer under our control can prove that stream is not losing messages and messages are ordered.