Patch timestamp architecture #1583

dimalit · 2023-07-06T18:33:26Z

Need to create some smart solution for backward compatibility: when schain is upgraded to new skaled version, this version should at first work as previous one - and after all have been upgraded - start to work in a new way. Currently, we use ContractStorageZeroValuePatch, ContractStorageZeroValuePatch etc. Probably we need "magic transaction" solution.

Tasks

Give feedback

dimalit · 2024-01-24T17:18:00Z

Investigation

Context needed by patches

Common: current/nearest block numbers/timestamps

ContractStorageLimitPatch

Used in State.
Context needed: none
Can be used as Ethereum fork.

CorrectForkInPowPatch

Used in:

ClientBase::estimateGas
Transaction (external gas)

Context needed:

ChainParams (scheduleForBlockNumber)

POWCheckPatch

Used in:

SealEngine

Context needed:

transaction
Can be used as Ethereum fork.

PrecompiledConfigPatch

Used in:

Precompiled

Context needed:

config json(old) / ChainParams (new)

RevertableFSPatch

Used in:

State (flag for overlayFS "cache" usage)

Can be used as Ethereum fork.

SkipInvalidTransactionsPatch

Used in:

Block (transaction execution)
Blockchain (filtering txns in responses)
Eth.cpp (filtering txns in responses)

Context needed:

too wide or:
blocks and transaction receipts to decide if they are invalid

TotalStorageUsedPatch

Keeps old broken record in DB. New one is ignored as DB hash computation

Used in:

Blockchain (DB management)

Context needed:

1 full DB usage recomputing
2 DB and block number (to write it)

Uses magic transaction!

TODO Modules vs activation patches
TODO flags vs added logic

dimalit · 2024-01-29T16:37:22Z

4 types of "patches":

"Flag" patch. Have only boolean state: enabled/disabled, this state is read in 1-2 places in code.
"EVM" patch. Alters some parameters (e.g. instruction cost or instruction enable/disable) in EVM, these parameters are read from 1-2 places, ideally - without "if"-s.
"Module" patch - enables or disables some module that is external to the patch (e.g. OverlayFS)
"Hardcore" patch. Affects many places in code, that cannot be separated into module. Contains all (or almost all) logic that needs to be altered within itself.

Stages of skaled pipeline affected by patches:

Transaction execution (including EVM) (ContractStorageLimitPatch)
Transaction validation (CorrectForkInPowPatch, POWCheckPatch)
JSON-RPC (CorrectForkInPowPatch, SkipInvalidTransactionsPatch)
Txn/call execution - precompiled (PrecompiledConfigPatch)
Txn/call execution - FS (RevertableFSPatch)
Transaction list execution (SkipInvalidTransactionsPatch)
Queries to blockchain (SkipInvalidTransactionsPatch)
Block finalization (i.e. write to DB) (TotalStorageUsedPatch)

TODO Lost patches in *.cpp; Consensus gets patches too!

dimalit · 2024-02-08T16:23:52Z

Notes on current patches:

getConfigVariableUint256 precompiled should know what block it is working on to get proper PrecompiledConfigPatch state (has only latest context)
OverlayDB::commitStorageValues needs only latest ContractStorageZeroValuePatch state (has no context)
State::commit needs only latest StorageDestructionPatch state
State::rollback?

dimalit · 2024-02-22T19:43:51Z

Where latestBlockTimestamp is available:

envInfo
Executive->envInfo
ExtVm->envInfo
Block::m_previousBlock

dimalit · 2024-02-26T15:54:24Z

Tests for patches:

ContractStorageLimitPatch enabled in jsonrpc.cpp YES storage_limit_reverted
ContractStorageZeroValuePatch disabled in tests' config (jsonrpc.cpp) (hard to test NB test manually, used in test storage_limit_predeployed)
RevertableFSPatch YES (FilestorageCacheSuite)
StorageDestructionPatch NO - test manually with JsonRpcSuite/simple_contract and see Self-destructing#
POWCheckPatch enabled in config (jsonrpc.cpp), used in powTxnGasLimit (enabled only)
PushZeroPatch NO -> YES (LegacyVMSuite/InstructionSuite)
SkipInvalidTransactionsPatch YES (SkaleHostSuite)
PrecompiledConfigPatch set to 1 in ClientTest.cpp and PrecompiledTest.cpp (test PrecompiledTests/getConfigVariable relies on it)
CorrectForkInPowPatch YES simplePoWTransaction
VerifyDaSigsPatch - need to check manually!

dimalit · 2024-03-06T12:03:31Z

Subtask #1846

DmytroNazarenko · 2024-05-17T20:33:21Z

skaled: 3.19.0-beta.0

oleksandrSydorenkoJ · 2024-06-20T11:34:04Z

Verified on regression network

skalenetwork/schain:3.19.0-beta.1

1. ContractStorageLimitPatch 
usage before test 273053993
uploaded 1 MB data:
 'skaledDBUsage': {'blocks.db_disk_usage': 305353679, 'contractStorageUsed': 19102272, 'pieceUsageBytes': 224758460, 'state.db_disk_usage': 273102856}}
remove 1 MB data:
'skaledDBUsage': {'blocks.db_disk_usage': 309056911, 'contractStorageUsed': 19202240, 'pieceUsageBytes': 248255788, 'state.db_disk_usage': 272308793}}

2. RevertableFSPatch 
eth.getTransactionReceipt('0x8bd54ccc5fb229ec9759795d83e8c5701dc069876fe6b70e0be34f697c897361')
  revertReason: "EVM revert instruction without description message",

3. ContractStorageZeroValuePatch
    2024-06-06 16:25:09.039346   Self-destruct cleared values:4

4. StorageDestructionPatch   
2024-06-06 16:25:09.039306   Self-destructing#4bf41edf…
2024-06-06 16:25:09.039315   Iterating over the LevelDB prefix: K��)$\v���ۼ��1�B�


5. PushZeroPatch            
2024-06-07 11:46:51.688132   http://localhost:46230 >>> {"id":2406556197,"jsonrpc":"2.0","method":"eth_call","params":[{"data":"0x58fa63ca","from":"0x71cBE3fedE33905d4D1Bf2Bd51f9d4A62375e659","to":"0x723d42fcB58d0007a179EcdCE8CF2921803ce7F5","value":"0x0"},"0x3a533"]}
2024-06-07 11:46:51.692009   http://localhost:46230 <<< {"id":2406556197,"jsonrpc":"2.0","result":"0x0000000000000000000000000000000000000000000000000000000000000000"}

6.POWCheckPatch 
7.correctForkInPowPatchTimestamp: 
8. SkipInvalidTransactionsPatch 
tests/skaled/functional/test_pow.py::test_0_gas_price Calculation gasPrice time is  2771.0483407974243
Block gas limit reached.
                                                                       
2024-06-06 17:29:45.790332   Transaction b438b900d0df04a9e974dc33a657e76c75b09cf6c95e84715188e0af26da9ec3 WouldNotBeInBlock: /home/s5/actions-runner-1/_work/skaled/skaled/libethereum/Executive.cpp(221): Throw in function static void dev::eth::Executive::verifyTransaction(const dev::eth::Transaction&, time_t, const dev::eth::BlockHeader&, const skale::State&, const dev::eth::ChainOperationParams&, const u256&, const u256&, bool)

9. verifyDaSigsPatchTimestamp
[2024-06-12 15:24:23.225] [3:main] [info] 808:CATCHUP_PROCESSED_BLOCKS:COUNT:1:DTM:29:PTM:4:LTM:0

[2024-06-12 15:24:34.131] [3:main] [info] 808:BLOCK_COMMITED: PRPSR:10:BID: 809:ROOT:96495378258348135281665955476550266297478359934801130133286416347315997853065:HASH:349f7e26:BLOCK_TXS:0:DMSG:0:TPRPS:7:MPRPS:4:RPRPS:1:TXS:0:TXLS:6:MGS:175:INSTS:285:BPS:0:HDRS:133:SOCK:0:FDS:407:PRT:8556:BTA:11121:BSA:0:TPS:0:LWT:0:LRT:0:LWC:14105:LRC:30990:KNWN:0:CONS:0:DSDS:384:SET:77:SBT:15:SEC:809:SBC:923:ZSC:1:EPT:2:STAMP:1718205871.539


10. precompiledConfigPatchTimestamp
0xb76A448071Ed77d22cAa1669567B5D28f5448d99
deployed oracle verif  
0x1bd4f059a8a55fb19b5b7df41924a1427615383f4a7b4c54bebb7e6dce349e7d 
check oracle vefifier   
0x86a1e58841e97b48639848da9de07c7d1ff75255d31907e6be905aa7680d2088
regression

11. EIP1559TransactionsPatchTimestamp
eth.getTransactionReceipt('0x399598b007bb252528628a6323eeafa433b255997fd1b5fc8ab2262af7cbc1cc').type
"0x0"

12. fastConsensusPatchTimestamp
[2024-06-13 18:08:42.793] [3:main] [info] 343394:CONSENSUS_STARTED:PROPOSING: 1011111110101001
[2024-06-13 18:08:44.353] [3:main] [info] 343395:CONSENSUS_STARTED:PROPOSING: 0000100000000000
...
[2024-06-13 18:10:17.180] [3:main] [info] 343457:CONSENSUS_STARTED:PROPOSING: 0010000000000000
[2024-06-13 18:10:18.168] [3:main] [info] 343458:CONSENSUS_STARTED:PROPOSING: 0001000000000000
[2024-06-13 18:10:19.962] [3:main] [info] 343459:CONSENSUS_STARTED:PROPOSING: 1011111110101001
[2024-06-13 18:10:21.719] [3:main] [info] 343460:CONSENSUS_STARTED:PROPOSING: 0000010000000000


[2024-06-13 18:11:54.284] [3:main] [info] 343523:CONSENSUS_STARTED:PROPOSING: 0000100000000000
[2024-06-13 18:11:56.556] [3:main] [info] 343524:CONSENSUS_STARTED:PROPOSING: 1011111110101001
[2024-06-13 18:11:58.621] [3:main] [info] 343525:CONSENSUS_STARTED:PROPOSING: 0000001000000000

13. flexibleDeploymentPatchTimestamp++


14. verifyBlsSyncPatchTimestamp
[2024-06-04 12:12:23.220] [2:main] [info] 0:SGX Enabled:0
[2024-06-04 12:12:23.220] [2:main] [info] 0:Is Sync Node:1
[2024-06-04 12:12:23.220] [2:main] [info] 0:Verify real signatures:1

dimalit self-assigned this Jul 6, 2023

dimalit added this to SKALE Engineering 🚀 Jul 6, 2023

DmytroNazarenko added the Technical debt label Jul 6, 2023

DmytroNazarenko moved this to To Do in SKALE Engineering 🚀 Jul 6, 2023

DmytroNazarenko added this to the SKALE 2.4 milestone Oct 13, 2023

PolinaKiporenko moved this from To Do to In Progress in SKALE Engineering 🚀 Jan 24, 2024

dimalit mentioned this issue Mar 4, 2024

Enhancement/skaled 1583 patch architecture #1842

Closed

PolinaKiporenko moved this from In Progress to Code Review in SKALE Engineering 🚀 Mar 11, 2024

dimalit mentioned this issue Apr 10, 2024

Enhancement/skaled 1583 patch architecture plus 17.2 #1868

Merged

dimalit linked a pull request Apr 11, 2024 that will close this issue

Enhancement/skaled 1583 patch architecture plus 17.2 #1868

Merged

dimalit moved this from Code Review to Ready For Release Candidate in SKALE Engineering 🚀 Apr 25, 2024

DmytroNazarenko closed this as completed May 16, 2024

DmytroNazarenko moved this from Ready For Release Candidate to Merged To Release Candidate in SKALE Engineering 🚀 May 17, 2024

EvgeniyZZ moved this from Merged To Release Candidate to QA in SKALE Engineering 🚀 May 20, 2024

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Patch timestamp architecture #1583

Patch timestamp architecture #1583

dimalit commented Jul 6, 2023 •

edited by PolinaKiporenko

Loading

Tasks

dimalit commented Jan 24, 2024 •

edited

Loading

dimalit commented Jan 29, 2024 •

edited

Loading

dimalit commented Feb 8, 2024

dimalit commented Feb 22, 2024 •

edited

Loading

dimalit commented Feb 26, 2024 •

edited

Loading

dimalit commented Mar 6, 2024

DmytroNazarenko commented May 17, 2024

oleksandrSydorenkoJ commented Jun 20, 2024

Patch timestamp architecture #1583

Patch timestamp architecture #1583

Comments

dimalit commented Jul 6, 2023 • edited by PolinaKiporenko Loading

Tasks

dimalit commented Jan 24, 2024 • edited Loading

Investigation

Context needed by patches

ContractStorageLimitPatch

CorrectForkInPowPatch

POWCheckPatch

PrecompiledConfigPatch

RevertableFSPatch

SkipInvalidTransactionsPatch

TotalStorageUsedPatch

dimalit commented Jan 29, 2024 • edited Loading

dimalit commented Feb 8, 2024

dimalit commented Feb 22, 2024 • edited Loading

dimalit commented Feb 26, 2024 • edited Loading

dimalit commented Mar 6, 2024

DmytroNazarenko commented May 17, 2024

oleksandrSydorenkoJ commented Jun 20, 2024

dimalit commented Jul 6, 2023 •

edited by PolinaKiporenko

Loading

dimalit commented Jan 24, 2024 •

edited

Loading

dimalit commented Jan 29, 2024 •

edited

Loading

dimalit commented Feb 22, 2024 •

edited

Loading

dimalit commented Feb 26, 2024 •

edited

Loading