Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

BC-7905 - Update documentation on tldraw #42

Merged
merged 23 commits into from
Dec 10, 2024
Merged
Show file tree
Hide file tree
Changes from 21 commits
Commits
Show all changes
23 commits
Select commit Hold shift + click to select a range
88697b5
BC-7905 - fix local setup
bergatco Nov 28, 2024
91110c3
BC-7905 - update config values
bergatco Nov 29, 2024
99e1c58
BC-7905 - update documentation for tldraw configuration and local setup
bergatco Dec 2, 2024
e64ab1f
BC-7905 - update technical documentation for tldraw backend architect…
bergatco Dec 2, 2024
f83cfd0
BC-7905 - update documentation for tldraw data storage and connection…
bergatco Dec 2, 2024
1b76dae
BC-7905 - move tldraw docs to separate folder / repository
bergatco Dec 2, 2024
9526d81
BC-7905 - add tldraw-server link to documentation navigation
bergatco Dec 2, 2024
e438a57
BC-7905 - fix images used for tldraw-server
bergatco Dec 2, 2024
a798d4c
BC-7905 - simplify file deletion process in tldraw documentation
bergatco Dec 3, 2024
64c2763
BC-7905 - change drawings
bergatco Dec 3, 2024
0869203
Merge branch 'main' into BC-7905-update-documentation-on-tldraw
bergatco Dec 3, 2024
49b0ab3
BC-7905 - refine how it works
bergatco Dec 4, 2024
df3d922
Merge branch 'BC-7905-update-documentation-on-tldraw' of https://gith…
bergatco Dec 4, 2024
a16f60d
BC-7905 - refine local setup
bergatco Dec 4, 2024
dd3e1fc
BC-7905 - refine technical details
bergatco Dec 4, 2024
f08cd1f
BC-7905 - refine asset upload documentation for tldraw
bergatco Dec 6, 2024
d443940
BC-7905 - refine documentation on how tldraw works and clarify termin…
bergatco Dec 9, 2024
9eb9dd9
BC-7905 - refine technical details in tldraw-server documentation to …
bergatco Dec 9, 2024
7e69a3f
BC-7905 - refine documentation on file upload process in tldraw white…
bergatco Dec 9, 2024
d877d76
Merge branch 'main' into BC-7905-update-documentation-on-tldraw
bergatco Dec 9, 2024
dc8819c
Merge branch 'main' into BC-7905-update-documentation-on-tldraw
SevenWaysDP Dec 9, 2024
9f36cb1
BC-7905 - replace Y-Redis with Redis
bergatco Dec 10, 2024
3692db0
Merge branch 'BC-7905-update-documentation-on-tldraw' of https://gith…
bergatco Dec 10, 2024
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
81 changes: 0 additions & 81 deletions docs/services/tldraw/How it works.md

This file was deleted.

30 changes: 0 additions & 30 deletions docs/services/tldraw/Local setup.md

This file was deleted.

90 changes: 0 additions & 90 deletions docs/services/tldraw/Technical details.md

This file was deleted.

Binary file removed docs/services/tldraw/assets/Create TLDRAW.drawio.png
Binary file not shown.
4 changes: 0 additions & 4 deletions docs/services/tldraw/assets/Create TLDRAW.drawio.svg

This file was deleted.

Binary file removed docs/services/tldraw/assets/Delete TLDRAW.drawio.png
Binary file not shown.
4 changes: 0 additions & 4 deletions docs/services/tldraw/assets/Delete TLDRAW.drawio.svg

This file was deleted.

Binary file removed docs/services/tldraw/assets/Use tldraw.drawio.png
Binary file not shown.
4 changes: 0 additions & 4 deletions docs/services/tldraw/assets/Use tldraw.drawio.svg

This file was deleted.

93 changes: 93 additions & 0 deletions docs/tldraw-server/How it works.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,93 @@
# How it works

bergatco marked this conversation as resolved.
Show resolved Hide resolved
The terms Redis and Valkey are used here synonymously and should describe the used in-memory-database.

## Configuration

- AUTHORIZATION_API_HOST - host address of the authorization endpoint (schuldcloud-server)
- FEATURE_TLDRAW_ENABLED - flag determining if tldraw is enabled
- LOGGER_LOG_LEVEL - logging level
- LOGGER_EXIT_ON_ERROR - flag whether an error will cause the application to stop
- METRICS_COLLECT_DEFAULT - flag whether the default metrics shall be collected
- REDIS_CLUSTER_ENABLED - flag whether a redis cluster or used or not
- REDIS_URL - redis connection string
- REDIS_SENTINEL_SERVICE_NAME - name of the redis sentinel service
- REDIS_PREFIX - prefix to be used with redis database
- REDIS_SENTINEL_NAME - name of the redis sentinel
- REDIS_SENTINEL_PASSWORD - password for the redis sentinel
- S3_ACCESS_KEY - access key for S3 storage
- S3_BUCKET - name of the S3 bucket
- S3_ENDPOINT - URL of the S3 service
- S3_PORT - port number for the S3 service
- S3_SECRET_KEY - secret key for S3 storage
- S3_SSL - flag to enable or disable SSL for S3 storage
- TLDRAW_ASSETS_ENABLED - enables uploading assets to tldraw board
- TLDRAW_ASSETS_MAX_SIZE_BYTES - maximum asset size in bytes
- TLDRAW_ASSETS_ALLOWED_MIME_TYPES_LIST - list of allowed assets MIME types
- TLDRAW_WEBSOCKET_PATH - path for the tldraw websocket connection
- TLDRAW_WEBSOCKET_URL - URL for the tldraw websocket connection
- WORKER_MIN_MESSAGE_LIFETIME - minimal lifetime of a update message consumed by the worker
- WORKER_TASK_DEBOUNCE - minimum idle time (in milliseconds) of the pending task messages to be claimed
- WORKER_TRY_CLAIM_COUNT - the maximum number of task messages to claim
- X_API_ALLOWED_KEYS - list of allowed xAPI keys

In order to have deletion functionality fully working locally you have to fill those feature flags, e.g.:

tldraw-server :
- X_API_ALLOWED_KEYS="7ccd4e11-c6f6-48b0-81eb-abcdef123456"

schulcloud-server :
- TLDRAW_ADMIN_API_CLIENT_API_KEY="7ccd4e11-c6f6-48b0-81eb-abcdef123456"
- TLDRAW_ADMIN_API_CLIENT_BASE_URL="http://localhost:3349"

## Create
![Create tldraw workflow](./assets/Create_TLDRAW.drawio.svg)

The Tldraw board can be created by the user on the courses ColumnBoard. It has a representation in ColumnBoard as DrawingElement inside a card (BoardNode in db). After creating representation as DrawingElement we can enter actual Tldraw SPA client on click.

1. User enters ColumnBoard and creates Representation of whiteboard (tldraw) in Card.
2. Data is saved and feedback with proper creation is given - user can see Representation and can enter whiteboard.
3. By entering whiteboard user is redirected to SPA tldraw-client.
4. Tldraw-client is starting WS connection with tldraw-server.
5. Tldraw-server first checks if user has permission to this resource (by checking if user has a permission to Representation of whiteboard - BoardNode).
Id of Representation is same as drawingName, which is visible in tldraw-client url.
6. If user has permission tldraw-server is allowing to remain connected and getting drawing data from S3 storage. If there is no drawing data available, the tldraw-server will create a new document automatically.

## Usage
![Usage tldraw workflow](./assets/Use_TLDRAW.drawio.svg)

### Connection

1. User joins tldraw board.
2. Tldraw-client connects to one of the tldraw-server pods and tries to establish websocket connection.
3. Tldraw-server calls schulcloud-server via HTTP requests to check user permissions. If everything is fine, the websocket connection is established.
4. Tldraw-server gets stored tldraw board data from S3 storage and sends it via websocket to connected user.
5. Tldraw-server starts subscribing to Redis PUBSUB channel corresponding to tldraw board name to listen to changes from other pods.

### Sending updates/storing data

1. Tldraw-client sends user's drawing changes to the tldraw-server via websocket connection.
2. Tldraw-server stores the board update in the valkey db - basically creates a diff between what's already stored and what's being updated.
3. Tldraw-server pushes the update to the boards Redis channel so that connected clients on different pods have synchronized board data.
4. Other pods subscribing to Redis channel send updates to their connected clients via websocket whenever they see a new message on Redis channel.
5. Finally the worker will run and persist the current state of the drawing data by applying all currently available updates from valkey on top of the stored drawing data and update the S3 storage accordingly.

## Delete
![Delete tldraw workflow](./assets/Delete_TLDRAW.drawio.svg)

1. User from schulcloud app in ColumnBoard deletes whiteboard (tldraw) instance form Card.
2. Schulcloud-server is removing representation data in schulcloud-database - BoardNodes collection.
3. Schulcloud-server is calling tldraw-server to delete all data that has given id.
4. Tldraw-server sends a delete action via websocket to inform connected clients about deletion. Clients redirect away from Tldraw-board to ensure that no new messages are added to valkey database.
5. Finally the worker will run, clear all updates and data from the valkey db and delete the drawing data from the S3 storage.

## Assets
### files upload

Images/GIFs can be uploaded into tldraw whiteboard by every user with access to the board. We use SVS FileStorageService to physically store uploaded assets while tldraw only holds URL to a resource.

The files are uploaded by calling schulcloud-api's fileController upload endpoint. This is possible because tldraw is represented as a boardnode called drawing-element. Mongo id of this drawing-element is a roomId used in URL param when connecting to a specific board.

### files deletion

The deletion of files is handled directly by the tldraw-client itself. On deletion in the UI, the client sends a delete request to the file storage. While awaiting the answer from file storage the editing of the Tldraw-board is blocked to prevent race conditions to the file storage.
Loading
Loading