Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Feature: connector should handle TRUNCATE messages from the server #168

Open
lyuboxa opened this issue Jun 11, 2024 · 0 comments
Open

Feature: connector should handle TRUNCATE messages from the server #168

lyuboxa opened this issue Jun 11, 2024 · 0 comments
Labels
feature New feature or request

Comments

@lyuboxa
Copy link
Contributor

lyuboxa commented Jun 11, 2024

Feature description

Currently, TRUNCATE messages are ignored. This can provide unexpected behaviour because the stream will not capture all records which have been deleted by the operation.

Each truncate message, contains the number of rels (tables) being truncated in the same LSN. This is tricky because:

  • The truncate message does not carry the keys of each row which needs to be deleted
  • It all happens in the same LSN
  • Conduit only supports Snapshot, Create, Delete and Update operations. Truncate does not fit in any of these

Some systems ignore it, others (like dbz) have an additional truncate operation to handle this type of operation.

Since its introduction in 2008, TRUNCATE is an optional feature in SQL. However, other systems do support way to empty a collection entirely without bother to emit individual event per delete.

The least work involved here will be to warn/block/something so the user is aware this is happening.

@lyuboxa lyuboxa added feature New feature or request triage Needs to be triaged labels Jun 11, 2024
@lovromazgon lovromazgon removed the triage Needs to be triaged label Jun 17, 2024
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
feature New feature or request
Projects
None yet
Development

No branches or pull requests

2 participants