-
Notifications
You must be signed in to change notification settings - Fork 214
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
[Merged by Bors] - sync2: add sqlstore #6405
Closed
Closed
Commits on Oct 23, 2024
-
sync2: improve rangesync Receive semantics
Given that after recent item sync is done (if it's needed at all), the range set reconciliation algorithm no longer depends on newly received item being added to the set, we can save memory by not adding the received items during reconciliation. During real sync, the received items will be sent to the respective handlers and after the corresponding data are fetched and validated, they will be added to the database, without the need to add them to cloned OrderedSets which are used to sync against particular peers.
Configuration menu - View commit details
-
Copy full SHA for 86b9591 - Browse repository at this point
Copy the full SHA 86b9591View commit details -
sync2: implement multi-peer synchronization
This adds multi-peer synchronization support. When the local set differs too much from the remote sets, "torrent-style" "split sync" is attempted which splits the set into subranges and syncs each sub-range against a separate peer. Otherwise, the full sync is done, syncing the whole set against each of the synchronization peers. Full sync is also done after each split sync run. The local set can be considered synchronized after the specified number of full syncs has happened. The approach is loosely based on [SREP: Out-Of-Band Sync of Transaction Pools for Large-Scale Blockchains](https://people.bu.edu/staro/2023-ICBC-Novak.pdf) paper by Novak Boškov, Sevval Simsek, Ari Trachtenberg, and David Starobinski.
Configuration menu - View commit details
-
Copy full SHA for ef30f47 - Browse repository at this point
Copy the full SHA ef30f47View commit details -
The `sqlstore` package provides simple sequence-based interface to the tables being synchronized. It is used by the FPTree data structure as the database layer, and doesn't do range fingerprinting by itself. `SyncedTable` and `SyncedTableSnapshot` provide methods that wrap the necessary SQL operations. `sql/expr` package was added to facilitate SQL generation.
Configuration menu - View commit details
-
Copy full SHA for bb43161 - Browse repository at this point
Copy the full SHA bb43161View commit details
Commits on Oct 26, 2024
-
Configuration menu - View commit details
-
Copy full SHA for af95c7f - Browse repository at this point
Copy the full SHA af95c7fView commit details -
Configuration menu - View commit details
-
Copy full SHA for b062eaa - Browse repository at this point
Copy the full SHA b062eaaView commit details
Commits on Oct 28, 2024
-
Configuration menu - View commit details
-
Copy full SHA for 3bfaa3f - Browse repository at this point
Copy the full SHA 3bfaa3fView commit details
Commits on Oct 30, 2024
-
Configuration menu - View commit details
-
Copy full SHA for ef484c5 - Browse repository at this point
Copy the full SHA ef484c5View commit details -
Configuration menu - View commit details
-
Copy full SHA for 0cf1678 - Browse repository at this point
Copy the full SHA 0cf1678View commit details -
Configuration menu - View commit details
-
Copy full SHA for 287d76d - Browse repository at this point
Copy the full SHA 287d76dView commit details -
Configuration menu - View commit details
-
Copy full SHA for 51166fa - Browse repository at this point
Copy the full SHA 51166faView commit details
Commits on Oct 31, 2024
-
Configuration menu - View commit details
-
Copy full SHA for c58d690 - Browse repository at this point
Copy the full SHA c58d690View commit details -
Configuration menu - View commit details
-
Copy full SHA for cf46587 - Browse repository at this point
Copy the full SHA cf46587View commit details -
Configuration menu - View commit details
-
Copy full SHA for 79eb4db - Browse repository at this point
Copy the full SHA 79eb4dbView commit details -
Configuration menu - View commit details
-
Copy full SHA for 9442698 - Browse repository at this point
Copy the full SHA 9442698View commit details
Commits on Nov 1, 2024
-
Configuration menu - View commit details
-
Copy full SHA for 8a8c60c - Browse repository at this point
Copy the full SHA 8a8c60cView commit details -
Configuration menu - View commit details
-
Copy full SHA for 9d6441f - Browse repository at this point
Copy the full SHA 9d6441fView commit details -
Configuration menu - View commit details
-
Copy full SHA for 6424aca - Browse repository at this point
Copy the full SHA 6424acaView commit details
Commits on Nov 2, 2024
-
sync2: sqlstore: cache generated SQL
Re-generating SQL statements every time is quick but increases GC pressure.
Configuration menu - View commit details
-
Copy full SHA for 79a31b7 - Browse repository at this point
Copy the full SHA 79a31b7View commit details -
Configuration menu - View commit details
-
Copy full SHA for 59f7db5 - Browse repository at this point
Copy the full SHA 59f7db5View commit details
Commits on Nov 3, 2024
-
Configuration menu - View commit details
-
Copy full SHA for 3750cb7 - Browse repository at this point
Copy the full SHA 3750cb7View commit details
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.