Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[Bug]: Messages loss on restarts #75

Open
limitium opened this issue Nov 6, 2024 · 0 comments · May be fixed by #76
Open

[Bug]: Messages loss on restarts #75

limitium opened this issue Nov 6, 2024 · 0 comments · May be fixed by #76
Labels
bug Something isn't working

Comments

@limitium
Copy link

limitium commented Nov 6, 2024

Bug Description

Current implementation losses messages, because SolaceTask#commit() method in reality doesn't have guarantee that all messages from the previous poll() call committed. This is explicitly highlighted in kafka-connect-api 3.6.1

"This method is invoked periodically when offsets are committed for this source task. Note that the offsets being committed won't necessarily correspond to the latest offsets returned by this source task via poll()."

Expected Behavior

Ack only really committed messages

Steps to Reproduce

Problem scenario:

  1. Connect calls poll() which returns, let's say 100 record
  2. Connect calls commit(), after partial message publishing 30
    a. Current implementation acks all 100
  3. Restart connect and 70 msgs will be lost

Solace Broker version

No response

Solace API

No response

Solace API version

No response

@limitium limitium added the bug Something isn't working label Nov 6, 2024
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
bug Something isn't working
Projects
None yet
1 participant