GC of clients and deleted records on server

[tslocke]

(I see the github discussions are not in use at all. Personally I find the forum-style structure works way better for tech help than the Discord shared-stream-of-consciousness, so I thought I'd start something here and see how it goes)

The server setup for my Replicache app has a few features that I think must be pretty common:

• A "workspace" (aka board/room/boundary of collaboration) with a single increasing integer version
• The db records that constitute the state of the workspace each also have a version - the version when that record was last updated
• The Replicache cookie is just the version number
• A clients table, tracking last seen mutation ID for each client

This makes the pull implementation pretty easy - send a 'put' for all records in the workspace that have version >= cookie.version.

But what about deletes? My solution, which I'm guessing is pretty standard, is to mark deleted, instead of actually deleting. No change to the way pull finds the records, just send 'del' instead of 'put' for matches that are marked deleted.

This raises a question about GC. In principle, once every client knows about the delete, the record can be deleted for real. An obvious way to track that would be to store the last sent cookie.version in the clients table. I can delete records that are marked deleted with a version number lower than any client has.

But clients are fairly ephemeral. A client might disappear having missed a delete and never return. That record would be stuck forever. As would the client record itself.

So this is related to GC of clients. Is it advisable to delete them after some time of inactivity?

Any advice appreciated. I guess one solution is "don't even worry about it" : )

[aboodman]

Yes your setup is very standard - it's exactly what we implemented in replicache-todo sample, for example.

Frequently people don't want to delete so-called "soft-deletes" for business/ux reasons. It might be useful to bring that data back from the dead for a variety of reasons.

If you do want to delete it, it makes sense to use the same policy you use for deleting clients. If you decide to delete clients older than, say, 1 month, then that means that clients can only go offline for 1 month. After that when they reconnect they will be unrecognized and will have to sync from scratch, losing any changes they made while offline.

With such a policy you could also safely hard-delete any soft-deleted records after one month because the only reason to keep soft deletes is to send them to clients. Once all clients that could conceivably need a delete are removed, the delete can also be removed.

We've never implemented this so I might be missing something, but it makes sense to me right now.

[aboodman]

(I would probably not delete clients or soft-deletes myself until my application was very large).