Feature/migrate modification history #2260
Open
Conversation
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
acerone85
requested review from
xgreenx,
Dentosal and
MitchTurner
as code owners
13 tasks
rafal-ch
reviewed
Sep 27, 2024
rafal-ch
reviewed
Sep 27, 2024
bin/fuel-core/src/cli/run.rs
Outdated
Comment on lines
640
to
637
| while let Err(e) = | ||
| rocks_db.migrate_modifications_history_at_height(height) | ||
| { | ||
| tracing::warn!("Could not migrate modifications history at height {height}: {e:?}. Trying again.") | ||
| } | ||
| } | ||
| } | ||
| rocks_db.complete_migration(); |
There was a problem hiding this comment.
In case of Err we will just warn in log and continue with next entry, and then call complete_migration();.
I'm wondering if this leaves some entries not migrated? And if this is acceptable outcome.
I may be lacking some context, though.
There was a problem hiding this comment.
That's a good question. I also lack context to answer, probably @xgreenx knows
There was a problem hiding this comment.
Ah, I missed the point of your question. The rationale behind the loop is because I don't know whether we use pessmiistic or optimistic concurrency control, e.g. whether the migration transaction aborts in case of conflict. In the latter case, we need to retry the transaction. But if we employ pessimistic concurrency control then the loop is not necessary
There was a problem hiding this comment.
I think we need to skip failed entry and continue. But with changes that I asked from #2233 (comment), you will not have cases where error can occur.
acerone85
force-pushed
the
feature/migrate-modification-history
branch
from
a29e3a8
to
e671b0c
Compare
acerone85
force-pushed
the
feature/version-modifications-history
branch
from
910186c
to
fe3e453
Compare
acerone85
force-pushed
the
feature/migrate-modification-history
branch
2 times, most recently
from
5c70a37
to
a3ecd66
Compare
xgreenx
reviewed
Oct 1, 2024
bin/fuel-core/src/cli/run.rs
Outdated
| // TODO: Handle graceful shutdown of migration processes | ||
| #[cfg(feature = "rocksdb")] | ||
| { | ||
| let on_chain_migration_handle = combined_database |
There was a problem hiding this comment.
I think the HistoricalRocksDB should be responsible for launching the migration on construction.
| @@ -329,10 +352,14 @@ where | |||
| fn rollback_block_to(&self, height_to_rollback: u64) -> StorageResult<()> { | |||
| let mut storage_transaction = self.db.read_transaction(); | |||
|
|
|||
| let modifications_history_migration_in_progress = self | |||
There was a problem hiding this comment.
Looks like would be nice to have a getter for this field=)
bin/fuel-core/src/cli/run.rs
Outdated
Comment on lines
640
to
637
| while let Err(e) = | ||
| rocks_db.migrate_modifications_history_at_height(height) | ||
| { | ||
| tracing::warn!("Could not migrate modifications history at height {height}: {e:?}. Trying again.") | ||
| } | ||
| } | ||
| } | ||
| rocks_db.complete_migration(); |
There was a problem hiding this comment.
I think we need to skip failed entry and continue. But with changes that I asked from #2233 (comment), you will not have cases where error can occur.
acerone85
force-pushed
the
feature/version-modifications-history
branch
from
8d4bd59
to
9e6247a
Compare
acerone85
force-pushed
the
feature/migrate-modification-history
branch
2 times, most recently
from
943f60c
to
0ff920e
Compare
acerone85
force-pushed
the
feature/version-modifications-history
branch
from
9e6247a
to
f787cf6
Compare
acerone85
force-pushed
the
feature/migrate-modification-history
branch
2 times, most recently
from
8ac1bf0
to
db2ffc5
Compare
acerone85
force-pushed
the
feature/version-modifications-history
branch
from
f787cf6
to
308c34a
Compare
acerone85
force-pushed
the
feature/migrate-modification-history
branch
from
7c1e835
to
e1bc889
Compare
acerone85
force-pushed
the
feature/version-modifications-history
branch
from
308c34a
to
3c9eff4
Compare
acerone85
force-pushed
the
feature/migrate-modification-history
branch
from
e1bc889
to
14540b7
Compare
acerone85
force-pushed
the
feature/migrate-modification-history
branch
from
14540b7
to
aff310a
Compare
acerone85
force-pushed
the
feature/migrate-modification-history
branch
from
aff310a
to
a3f9c25
Compare
AurelienFT
previously approved these changes
Oct 29, 2024
acerone85
force-pushed
the
feature/migrate-modification-history
branch
from
db51eeb
to
dbc1786
Compare
xgreenx
added a commit
that referenced
this pull request
Nov 27, 2024
## Linked Issues/PRs <!-- List of related issues/PRs --> Related to #2095 ## Description <!-- List of detailed changes --> - [ ] Adds a ModificationsHistoryV2 column in the Historical RocksDB, with keys encoded in big endian - [ ] Adds a flag to the historical RocksDB that keeps track whether the migration is still in progress - [ ] Change HistoricalRocksDB to alway insert historical changes from V2 table. When the migration is in progress, delete key-value pairs from both V2 and V2, read key-value pairs from V2 falling back to V1 if no value in V2 is found #TODO (either this or next MR): - [x] Small task that performs the migration of key-value pairs from V1 to V2 (Done in #2260) ## Checklist - [ ] Breaking changes are clearly marked as such in the PR description and changelog - [x] New behavior is reflected in tests - [ ] [The specification](https://github.com/FuelLabs/fuel-specs/) matches the implemented behavior (link update PR if changes are needed) ### Before requesting review - [ ] I have reviewed the code myself - [ ] I have created follow-up issues caused by this PR and linked them here ### After merging, notify other teams [Add or remove entries as needed] - [ ] [Rust SDK](https://github.com/FuelLabs/fuels-rs/) - [ ] [Sway compiler](https://github.com/FuelLabs/sway/) - [ ] [Platform documentation](https://github.com/FuelLabs/devrel-requests/issues/new?assignees=&labels=new+request&projects=&template=NEW-REQUEST.yml&title=%5BRequest%5D%3A+) (for out-of-organization contributors, the person merging the PR will do this) - [ ] Someone else? --------- Co-authored-by: Green Baneling <[email protected]>
Base automatically changed from
feature/version-modifications-history
to
master
November 27, 2024 14:35
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
Linked Issues/PRs
Closes #2095
Description
Questions:
Checklist
Before requesting review
After merging, notify other teams
[Add or remove entries as needed]