Skip to content

Commit

Permalink
fix lint
Browse files Browse the repository at this point in the history
  • Loading branch information
Trisfald committed Nov 15, 2024
1 parent 615a92f commit e488308
Showing 1 changed file with 32 additions and 24 deletions.
56 changes: 32 additions & 24 deletions neps/nep-0568.md
Original file line number Diff line number Diff line change
Expand Up @@ -229,28 +229,31 @@ During a resharding event, at the boundary of the epoch, when we need to split t

Resharding Flat State is a time consuming operation and it runs in parallel with block processing for several block heights.
Thus, there are a few important aspects to consider during implementation:
- Flat State's own status should be resilient to application crashes.
- The parent shard's Flat State should be split at the correct block height.
- New shards' Flat States should eventually converge to same representation the chain is using to process blocks (MemTries).
- Resharding should work correctly in the presence of chain forks.
- Retired shards are cleaned up.

* Flat State's own status should be resilient to application crashes.
* The parent shard's Flat State should be split at the correct block height.
* New shards' Flat States should eventually converge to same representation the chain is using to process blocks (MemTries).
* Resharding should work correctly in the presence of chain forks.
* Retired shards are cleaned up.

Note that the Flat States of the newly created shards won't be available until resharding is completed. This is fine because the temporary MemTries are
built instantly and they can satisfy all block processing needs.

The main component responsible to carry out resharding on Flat State is the [FlatStorageResharder](https://github.com/near/nearcore/blob/f4e9dd5d6e07089dfc789221ded8ec83bfe5f6e8/chain/chain/src/flat_storage_resharder.rs#L68).

#### Flat State's status persistence

Every shard Flat State has a status associated to it and stored in the database, called `FlatStorageStatus`. We propose to extend the existing object
by adding the new enum variant named `FlatStorageStatus::Resharding`. This approach has two benefits. First, the progress of any Flat State resharding is
persisted to disk, which makes the operation resilient to a node crash or restart. Second, resuming resharding on node restart shares the same code path as Flat
State creation (see `FlatStorageShardCreator`), reducing the code duplication factor.

`FlatStorageStatus` is updated at every committable step of resharding. The commit points are the following:
- Beginning of resharding or, in other words, the last block of the old shard layout.
- Scheduling of the _"split parent shard"_ task.
- Execution, cancellation or failure of the _"split parent shard"_ task.
- Execution or failure of any _"child catchup"_ task.

* Beginning of resharding or, in other words, the last block of the old shard layout.
* Scheduling of the _"split parent shard"_ task.
* Execution, cancellation or failure of the _"split parent shard"_ task.
* Execution or failure of any _"child catchup"_ task.

#### Splitting a shard Flat State

Expand All @@ -269,30 +272,35 @@ One current technical limitation is that, upon a node crash or restart, the _"sp
A reference implementation of splitting a Flat State can be found in [FlatStorageResharder::split_shard_task](https://github.com/near/nearcore/blob/fecce019f0355cf89b63b066ca206a3cdbbdffff/chain/chain/src/flat_storage_resharder.rs#L295).

#### Values assignment from parent to child shards

Key-value pairs in the parent shard Flat State are inherited by children according to the rules stated below.

Elements inherited by the child shard which tracks the `account_id` contained in the key:
- `ACCOUNT`
- `CONTRACT_DATA`
- `CONTRACT_CODE`
- `ACCESS_KEY`
- `RECEIVED_DATA`
- `POSTPONED_RECEIPT_ID`
- `PENDING_DATA_COUNT`
- `POSTPONED_RECEIPT`
- `PROMISE_YIELD_RECEIPT`

* `ACCOUNT`
* `CONTRACT_DATA`
* `CONTRACT_CODE`
* `ACCESS_KEY`
* `RECEIVED_DATA`
* `POSTPONED_RECEIPT_ID`
* `PENDING_DATA_COUNT`
* `POSTPONED_RECEIPT`
* `PROMISE_YIELD_RECEIPT`

Elements inherited by both children:
- `DELAYED_RECEIPT_OR_INDICES`
- `PROMISE_YIELD_INDICES`
- `PROMISE_YIELD_TIMEOUT`
- `BANDWIDTH_SCHEDULER_STATE`

* `DELAYED_RECEIPT_OR_INDICES`
* `PROMISE_YIELD_INDICES`
* `PROMISE_YIELD_TIMEOUT`
* `BANDWIDTH_SCHEDULER_STATE`

Elements inherited only be the lowest index child:
- `BUFFERED_RECEIPT_INDICES `
- `BUFFERED_RECEIPT`

* `BUFFERED_RECEIPT_INDICES `
* `BUFFERED_RECEIPT`

#### Bring children shards up to date with the chain's head

Children shards' Flat States build a complete view of their content at the height of the `resharding block` sometime during the new epoch
after resharding. At that point in time many new blocks have been processed already, and these will most likely contain updates for the new shards. A catch-up step is necessary to apply all Flat State deltas accumulated until now.

Expand Down

0 comments on commit e488308

Please sign in to comment.