How to store all slots' beacon state

[hyunsooda]

With the option chain.archiveStateEpochFrequency 1, node stores states per 32 slots. Is there an alternative to store all slots?

Thank you.

[nflaig]

Currently there is no option to store a state per slot. What's your specific use case? I am guessing you want to retrieve the state (or other data that requires to load the state) for slot x. We have a draft PR (#6033) that will make it possible to access historical state data and does not require to store the full state each epoch / slot.

[hyunsooda]

Thank you. The node now successfully follows up on the latest slot, but some queries are still not retrievable. The following is a simple query script from BASE slot number to BASE-1000.

BASE=3934562
for i in {1..1000}
do
   curl -X 'GET' "http://3.37.16.97:9596/eth/v1/beacon/states/$((BASE - i))/root" -H 'accept: application/json'
   echo ""
done

{"data":{"root":"0x7c9276184ee508bb877399be46d217e26f1ccf8f68434b770e4147c5dabf44d4"},"execution_optimistic":false}
{"data":{"root":"0x4bf3fe59134afcfc70c0cb44855b527ed652208a0d10db3fa6a89ce572906bd0"},"execution_optimistic":false}
{"data":{"root":"0x5ad6d870190f17357fe9538c19f4ed2a8156c4e0555ee15051f157691f2b2e6c"},"execution_optimistic":false}
{"data":{"root":"0x275ebd8d4c23cacfccbc84ab33ed53f512a0d59ff626a82f1ac60e75490a1139"},"execution_optimistic":false}
{"data":{"root":"0x8a5575d13739447234581ff9ed73fe4be422cf441c9e8d9ab74a1608b6d2a8cc"},"execution_optimistic":false}
{"data":{"root":"0xf384e29df627a90ee9488610926e854e9a7f4319770afac9b1d70966fdb6a3c2"},"execution_optimistic":false}
{"data":{"root":"0x15e20add843b01a3bce85a3c0b41c7fb3de79e6f7fb486a111123ef897beadb2"},"execution_optimistic":false}
{"data":{"root":"0xf36ed05519b22088f4f7753bd95763784c0ce7e20f181e52e7cdf3e018894237"},"execution_optimistic":false}
{"data":{"root":"0x86963a62755a6ac0399a9b83ea04e2f44a20076e9219f813031109dd0420f496"},"execution_optimistic":false}
{"data":{"root":"0x8bdb767a6a3c6a0d697724ef8da1e93d75ba76715d57ba4aa8bfdcad13786388"},"execution_optimistic":false}
{"data":{"root":"0x09493bd8a79c19bf6e58f229df0ecbd2988f5b0f562cd9101fbb5b3eb5997280"},"execution_optimistic":false}
{"data":{"root":"0x7125910c4204f6f40431c66330faa576f226ccd089b59ac8580cb91e929731dc"},"execution_optimistic":false}
{"data":{"root":"0x8cd0621fc0a6f00b6fee72bd11975f8170e275a21822620021a2c3286f5d64e4"},"execution_optimistic":false}
{"data":{"root":"0x1e69fb69bd240ead4a4d1a13140eec12801b3695ac3a02572e8ffdd2db87d721"},"execution_optimistic":false}
{"data":{"root":"0xe42ed2e3dfd47067f1a8f790b6da4769022d303cf1e82a798031471e24be0c50"},"execution_optimistic":false}
{"data":{"root":"0x9850fe6662f348334989645d8462ae07a613318fb4fbd71401fa130165edbc7b"},"execution_optimistic":false}
{"data":{"root":"0x4567976d52125447255072b96044a6dbf89288f90bc2e800d650f6cbba8641f4"},"execution_optimistic":false}
{"data":{"root":"0xef0c8bfae392ae627af7f5d478df54b03d78dfffe3005fcee6a232cf3ee434c1"},"execution_optimistic":false}
{"data":{"root":"0x886e969f6e49b0f8921a4f20cc5571042b7daa70f7f09ca322160eea7afe6f7a"},"execution_optimistic":false}
{"data":{"root":"0x7b2aad8a1bc9b325cc2440411be1f630e5fa90c49aaf3d4d489f2261d3abcad2"},"execution_optimistic":false}
{"statusCode":404,"error":"Not Found","message":"No state found for id '3934541'"}
{"data":{"root":"0x8bead4d735cacc831eb08e1d73648d689a3285fc2bee071517b9f72c810e8730"},"execution_optimistic":false}
{"data":{"root":"0x3f05b63c4b3207e1d9e21e7c3c1fdab1880a3ae43528f27b8b2ca2150ba4b671"},"execution_optimistic":false}
{"data":{"root":"0x6d8aa591a893fc6c78a9e2913a9d78f95329a55c5214eb5c11af989c5bde57f4"},"execution_optimistic":false}
...
...
...

The node execution command is as follows:

./lodestar beacon \
  --network=sepolia \
  --dataDir="../lodestar-sepolia-datadir-regen" \
  --execution.urls="http://127.0.0.1:8551" \
  --jwt-secret=$JWT_PATH \
  --checkpointSyncUrl="https://lodestar-sepolia.chainsafe.io" \
  --rest.address="0.0.0.0" \
  --chain.archiveStateEpochFrequency 1 \
  --eth1.depositContractDeployBlock=0

I expected that the regeneration feature could respond against whatever the slot number is, but it seems not. Am I misunderstanding? or is there another way to resolve this? Thank you.

[nflaig]

I don't think it will be possible right now until #6033 is implemented. You should always be able to get the unfinalized state (last 2 epochs) but once the state is finalized we prune it from in-memory store and rely on reloading it from disk but even with --chain.archiveStateEpochFrequency 1 it does not persist the state for every slot, although you should be able to load the state on each epoch boundary / start slot in the epoch (epoch * 32).

[hyunsooda]

I executed my node using PR-6033. Were you suggesting that retrieval is consistently achievable at any slot number with PR-6033?

[wemeetagain]

Not yet, that PR isnt functional yet

[hyunsooda]

Got it. Le me close this now. Thank you @nflaig, @wemeetagain .

[nflaig]

Keeping the discussion open to make it easier for others to find in case the have a similar / related question. Github hides closed discussions by default