Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

BC-8431 - Broken tldraw docs when reloading #38

Merged
merged 19 commits into from
Nov 26, 2024
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
19 commits
Select commit Hold shift + click to select a range
b0f51c4
Handle pending structs in y-redis service to prevent premature docume…
SevenWaysDP Nov 20, 2024
1b5126d
BC-8431 - refactor message reading logic to use configurable counts f…
bergatco Nov 22, 2024
cbc2ce0
BC-8431 - refactor handleMessageUpdates to move variable declarations…
bergatco Nov 22, 2024
37b6445
BC-8431 - rnhance readMessagesFromStream to accept configurable count…
bergatco Nov 22, 2024
396989d
BC-8431 - refactor getDoc method to return undefined for non-existent…
bergatco Nov 25, 2024
3495505
BC-8431 - refactor worker service to improve document deletion logic …
bergatco Nov 25, 2024
3b07b4e
BC-8431 - add deleteMessagesFromStream method and update message dele…
bergatco Nov 25, 2024
8f73c99
BC-8431 - revert worker service and ws service to its former state
bergatco Nov 25, 2024
f32c63c
BC-8431 - remove no longer required console.logs
bergatco Nov 25, 2024
9ef35f8
BC-8431 - revert return type of getDoc of API service
bergatco Nov 25, 2024
95859eb
BC-8431 - revert worker as well as ws service some more
bergatco Nov 25, 2024
0065923
BC-8431 - improve console logs in API service and ioredis adapter
bergatco Nov 25, 2024
82ce14f
Merge branch 'main' into BC-8431
bergatco Nov 25, 2024
ad04269
BC-8431 - simplify readMessagesFromStream and getDoc methods by remov…
SevenWaysDP Nov 25, 2024
f931c9a
BC-8431 - refactor subscribe method to use map.setIfUndefined for cle…
SevenWaysDP Nov 25, 2024
4e62569
refactor RedisAdapter interface and improve logging in API service
SevenWaysDP Nov 25, 2024
872cd31
refactor readMessagesFromStream method to remove unnecessary blocking…
SevenWaysDP Nov 25, 2024
3f4a78a
BC-8431 - remove `deleteMessagesFromStream` from redis adapter
bergatco Nov 26, 2024
fe62a8c
BC-8431 - fix ioredis adapter tests
bergatco Nov 26, 2024
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
2 changes: 1 addition & 1 deletion src/infra/redis/ioredis.adapter.spec.ts
Original file line number Diff line number Diff line change
Expand Up @@ -255,7 +255,7 @@ describe(IoRedisAdapter.name, () => {
// @ts-ignore
const id = readBufferReply[0][0].toString();

const expectedProps = ['COUNT', 1000, 'BLOCK', 1000, 'STREAMS', computeRedisRoomStreamName, '0'];
const expectedProps = ['STREAMS', computeRedisRoomStreamName, '0'];

const expectedResult = [
{
Expand Down
10 changes: 1 addition & 9 deletions src/infra/redis/ioredis.adapter.ts
Original file line number Diff line number Diff line change
Expand Up @@ -105,15 +105,7 @@ export class IoRedisAdapter implements RedisAdapter {
}

public async readMessagesFromStream(streamName: string): Promise<StreamMessagesReply> {
const reads = await this.redis.xreadBuffer(
'COUNT',
1000, // Adjust the count as needed
'BLOCK',
1000, // Adjust the block time as needed
'STREAMS',
streamName,
'0',
);
const reads = await this.redis.xreadBuffer('STREAMS', streamName, '0');

const streamReplyRes = mapToStreamMessagesReply(reads);

Expand Down
9 changes: 7 additions & 2 deletions src/infra/y-redis/api.service.ts
Original file line number Diff line number Diff line change
Expand Up @@ -94,7 +94,6 @@ export class Api {

public async getDoc(room: string, docid: string): Promise<YRedisDoc> {
const end = MetricsService.methodDurationHistogram.startTimer();

let docChanged = false;

const roomComputed = computeRedisRoomStreamName(room, docid, this.redisPrefix);
Expand Down Expand Up @@ -123,13 +122,19 @@ export class Api {

end();

return {
const response = {
ydoc,
awareness,
redisLastId: docMessages?.lastId.toString() ?? '0',
storeReferences: docstate?.references ?? null,
docChanged,
};

if (ydoc.store.pendingStructs !== null) {
console.warn(`Document ${room} has pending structs ${JSON.stringify(ydoc.store.pendingStructs)}.`);
}

return response;
}

public async destroy(): Promise<void> {
Expand Down
7 changes: 2 additions & 5 deletions src/infra/y-redis/subscriber.service.ts
Original file line number Diff line number Diff line change
Expand Up @@ -5,6 +5,7 @@
The original code from the `y-redis` repository is licensed under the AGPL-3.0 license.
https://github.com/yjs/y-redis
*/
import * as map from 'lib0/map';
import { RedisService } from '../redis/redis.service.js';
import { Api, createApiClient } from './api.service.js';
import { isSmallerRedisId } from './helper.js';
Expand Down Expand Up @@ -50,13 +51,9 @@ export class Subscriber {
}

public subscribe(stream: string, f: SubscriptionHandler): { redisId: string } {
const sub = this.subscribers.get(stream) ?? { fs: new Set<SubscriptionHandler>(), id: '0', nextId: null };
const sub = map.setIfUndefined(this.subscribers, stream, () => ({ fs: new Set(), id: '0', nextId: null }));
sub.fs.add(f);

if (!this.subscribers.has(stream)) {
this.subscribers.set(stream, sub);
}

return {
redisId: sub.id,
};
Expand Down
Loading