sync2: multipeer: fix edge cases #6447
Open
Conversation
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Split sync could become blocked when there were slow peers. Their subranges are assigned to other peers, and there were bugs causing indefinite blocking and panics in these cases. Moreover, after other peers managed to sync the slow peers' subranges ahead of them, we need to interrupt syncing against the slow peers as it's no longer needed. In multipeer sync, when every peer has failed to sync, e.g. due to temporary connection interruption, we don't need to wait for the full sync interval, using shorter wait time between retries.
ivan4th
requested review from
dshulyak,
fasmat,
poszu,
acud and
jellonek
as code owners
Codecov ReportAttention: Patch coverage is
✅ All tests successful. No failed tests found. Additional details and impacted files@@ Coverage Diff @@
## sync2/dbset-conns #6447 +/- ##
===================================================
- Coverage 79.9% 79.8% -0.1%
===================================================
Files 352 352
Lines 46436 46497 +61
===================================================
+ Hits 37108 37145 +37
- Misses 7223 7249 +26
+ Partials 2105 2103 -2 ☔ View full report in Codecov by Sentry. |
## Motivation Most of the time, there are some nodes that already joined the P2P network but didn't complete their initialization yet, and thus attempts to e.g. fetch blobs to them end up in stream setup errors (protocol not supported). Additionally, during initial phases of `syncv2` testing, we're going to have a limited number of nodes supporting `sync/2` protocol, and when choosing peers for `syncv2`, we must only include these peers. Co-authored-by: Ivan Shvedunov <[email protected]>
## Motivation `SyncedTable` has a tiny cache for query strings to avoid heap-intensive query formatting operations (AST->SQL string) upon each SQL query being run. The cached query string map was not protected by a mutex.
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
Motivation
Split sync could become blocked when there were slow peers. Their
subranges are assigned to other peers, and there were bugs causing
indefinite blocking and panics in these cases. Moreover, after other
peers managed to sync the slow peers' subranges ahead of them, we need
to interrupt syncing against the slow peers as it's no longer needed.
In multipeer sync, when every peer has failed to sync, e.g. due to
temporary connection interruption, we don't need to wait for the full
sync interval, using shorter wait time between retries.
Description
This fixes aforementioned multipeer sync issues, and adds tests.
It also adds sync interval randomization to avoid network load spikes.