Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Sync from scratch - been two weeks #218

Closed
quakemmo opened this issue Sep 7, 2023 · 6 comments
Closed

Sync from scratch - been two weeks #218

quakemmo opened this issue Sep 7, 2023 · 6 comments
Assignees
@blxdyx

Comments

@quakemmo
Copy link

quakemmo commented Sep 7, 2023

System information

Erigon version: ./erigon --version
erigon version 2022.08.3-alpha-abb5c9a5

OS & Version: Windows/Linux/OSX
Linux

Commit hash:

Erigon Command (with flags/config):
./erigon

Started erigon on a server couple weeks ago. It downloaded about 2 Tb of data in a few days, now it has been taking all CPU for about a week, and logs only contain this:
EROR[09-07|08:26:10.648] Staged Sync err="[1/16 Headers] BuildMissedIndices: rlp: expected List"
INFO[09-07|08:26:11.168] [Snapshots] Fetching torrent files metadata
INFO[09-07|08:26:11.174] [Snapshots] Stat blocks=0k indices=0k alloc=7.3GB sys=15.7GB
EROR[09-07|08:26:11.592] Staged Sync err="[1/16 Headers] BuildMissedIndices: read ParentHash: rlp: element is larger than containing list"
INFO[09-07|08:26:12.398] [Snapshots] Fetching torrent files metadata
INFO[09-07|08:26:12.406] [Snapshots] Stat blocks=0k indices=0k alloc=8.0GB sys=15.7GB
EROR[09-07|08:26:12.625] Staged Sync err="[1/16 Headers] BuildMissedIndices: read ParentHash: rlp: element is larger than containing list"
INFO[09-07|08:26:13.155] [Snapshots] Fetching torrent files metadata
INFO[09-07|08:26:13.161] [Snapshots] Stat blocks=0k indices=0k alloc=7.8GB sys=15.7GB
EROR[09-07|08:26:14.186] Staged Sync err="[1/16 Headers] BuildMissedIndices: read ParentHash: rlp: element is larger than containing list"
INFO[09-07|08:26:15.145] [Snapshots] Fetching torrent files metadata
INFO[09-07|08:26:15.153] [Snapshots] Stat blocks=0k indices=0k alloc=8.2GB sys=15.7GB
EROR[09-07|08:26:16.048] Staged Sync err="[1/16 Headers] BuildMissedIndices: read ParentHash: rlp: element is larger than containing list"
INFO[09-07|08:26:16.559] [Snapshots] Fetching torrent files metadata
INFO[09-07|08:26:16.564] [Snapshots] Stat blocks=0k indices=0k alloc=7.4GB sys=15.7GB
EROR[09-07|08:26:16.921] Staged Sync err="[1/16 Headers] BuildMissedIndices: read ParentHash: rlp: element is larger than containing list"

It's been a week. I'm not sure there's anything advancing in the process. Thanks for any insights.
@blxdyx
Copy link
Collaborator

blxdyx commented Sep 11, 2023

start command can refer to #41

@blxdyx
Copy link
Collaborator

blxdyx commented Sep 11, 2023

Try start with --downloader.verify

@quakemmo
Copy link
Author

Hello, and thank you, @blxdyx, for your replies.

--downloader.verify didn't help much.

I start erigon with the following command line:
./erigon --chain=bsc --p2p.protocol=67 --db.pagesize=16k --txpool.disable --maxpeers=30 --verbosity=3 --metrics --downloader.verify --authrpc.port 18473 --http.port 18472

This is what logs look like, let it run for almost an hour:

INFO[09-11|12:11:40.592] Starting metrics server                  addr=http://127.0.0.1:6060/debug/metrics/prometheus
INFO[09-11|12:11:40.592] Build info                               git_branch=devel git_tag= git_commit=abb5c9a5c4bd701de5d7fb4fb9b6b0f8f7c167c8
INFO[09-11|12:11:40.592] Starting Erigon on BSC mainnet... 
INFO[09-11|12:11:40.594] Maximum peer count                       ETH=30 total=30
INFO[09-11|12:11:40.594] starting HTTP APIs                       APIs=eth,erigon,engine
INFO[09-11|12:11:40.594] torrent verbosity                        level=WRN
INFO[09-11|12:11:42.759] Set global gas cap                       cap=50000000
INFO[09-11|12:11:42.768] Opening Database                         label=chaindata path=/home/user/.local/share/erigon/chaindata
INFO[09-11|12:11:42.769] Initialised chain configuration          config="{ChainID: 1, Homestead: 1150000, DAO: 1920000, DAO Support: true, Tangerine Whistle: 2463000, Spurious Dragon: 2675000, Byzantium: 4370000, Constantinople: 7280000, Petersburg: 7280000, Istanbul: 9069000, Muir Glacier: 9200000, Berlin: 12244000, London: 12965000, Arrow Glacier: 13773000, Gray Glacier: 15050000, Terminal Total Difficulty: 58750000000000000000000, Merge Netsplit: <nil>, Engine: ethash}" genesis=0xd4e56740f876aef8c010b86a40d5f56745a118d0906a34e69aec8c0db1cb8fa3
INFO[09-11|12:11:42.769] Effective                                prune_flags= snapshot_flags="--snapshots=true" history.v2=false
INFO[09-11|12:11:43.248] Initialising Ethereum protocol           network=56
INFO[09-11|12:11:43.248] Disk storage enabled for ethash DAGs     dir=/home/user/.local/share/erigon/ethash-dags count=2
INFO[09-11|12:11:43.249] Starting private RPC server              on=127.0.0.1:9090
INFO[09-11|12:11:43.249] new subscription to logs established 
INFO[09-11|12:11:43.249] rpc filters: subscribing to Erigon events 
INFO[09-11|12:11:43.249] new subscription to newHeaders established 
INFO[09-11|12:11:43.249] Reading JWT secret                       path=/home/user/.local/share/erigon/jwt.hex
INFO[09-11|12:11:43.250] HTTP endpoint opened for Engine API      url=localhost:18473 ws=true ws.compression=true
INFO[09-11|12:11:43.250] HTTP endpoint opened                     url=localhost:18472 ws=false ws.compression=true grpc=false
INFO[09-11|12:11:43.259] [Snapshots] Fetching torrent files metadata 
INFO[09-11|12:11:43.373] Started P2P networking                   version=67 self=enode://<stripped>@127.0.0.1:30303 name=erigon/v2022.08.3-alpha-abb5c9a5/linux-amd64/go1.18.6
EROR[09-11|12:11:45.001] SOAP do: Post "http://192.168.1.<stripped>:1780/control?WANIPConnection": net/http: HTTP/1.x transport connection broken: malformed HTTP version "HTTP1.1" 
WARN[09-11|12:11:45.001] UPnP device at 192.168.1.<stripped>: mapping internal UDP port 42069: error: Post "http://192.168.1.<stripped>:1780/control?WANIPConnection": net/http: HTTP/1.x transport connection broken: malformed HTTP version "HTTP1.1" 
EROR[09-11|12:11:45.015] SOAP do: Post "http://192.168.1.<stripped>:1780/control?WANIPConnection": net/http: HTTP/1.x transport connection broken: malformed HTTP version "HTTP1.1" 
WARN[09-11|12:11:45.015] UPnP device at 192.168.1.<stripped>: mapping internal TCP port 42069: error: Post "http://192.168.1.<stripped>:1780/control?WANIPConnection": net/http: HTTP/1.x transport connection broken: malformed HTTP version "HTTP1.1" 
INFO[09-11|12:12:03.267] [snapshots] Verifying                    progress=15.06%
INFO[09-11|12:12:23.279] [snapshots] Verifying                    progress=19.45%
INFO[09-11|12:12:43.268] [snapshots] Verifying                    progress=23.36%
INFO[09-11|12:13:03.282] [snapshots] Verifying                    progress=27.29%
INFO[09-11|12:13:23.276] [snapshots] Verifying                    progress=31.14%
INFO[09-11|12:13:43.252] [p2p] GoodPeers                          eth67=0
INFO[09-11|12:13:43.287] [snapshots] Verifying                    progress=35.04%
INFO[09-11|12:14:03.263] [snapshots] Verifying                    progress=38.87%
INFO[09-11|12:14:23.265] [snapshots] Verifying                    progress=42.57%
INFO[09-11|12:14:43.281] [snapshots] Verifying                    progress=46.42%
INFO[09-11|12:15:03.286] [snapshots] Verifying                    progress=50.30%
INFO[09-11|12:15:23.296] [snapshots] Verifying                    progress=54.19%
INFO[09-11|12:15:43.254] [p2p] GoodPeers                          eth67=0
INFO[09-11|12:15:43.272] [snapshots] Verifying                    progress=58.11%
INFO[09-11|12:16:03.269] [snapshots] Verifying                    progress=61.90%
INFO[09-11|12:16:23.265] [snapshots] Verifying                    progress=65.78%
INFO[09-11|12:16:43.265] [snapshots] Verifying                    progress=69.67%
INFO[09-11|12:17:03.267] [snapshots] Verifying                    progress=73.46%
INFO[09-11|12:17:23.260] [snapshots] Verifying                    progress=77.06%
INFO[09-11|12:17:43.249] [p2p] GoodPeers                          eth67=0
INFO[09-11|12:17:43.261] [snapshots] Verifying                    progress=80.79%
INFO[09-11|12:18:03.277] [snapshots] Verifying                    progress=84.45%
INFO[09-11|12:18:23.261] [snapshots] Verifying                    progress=88.31%
INFO[09-11|12:18:43.261] [snapshots] Verifying                    progress=92.13%
INFO[09-11|12:19:03.267] [snapshots] Verifying                    progress=95.87%
INFO[09-11|12:19:23.267] [snapshots] Verifying                    progress=99.26%
INFO[09-11|12:19:35.912] [Snapshots] Stat                         blocks=0k indices=0k alloc=2.3GB sys=2.4GB
EROR[09-11|12:19:36.191] Staged Sync                              err="[1/16 Headers] BuildMissedIndices: read ParentHash: rlp: element is larger than containing list"
INFO[09-11|12:19:36.695] [Snapshots] Fetching torrent files metadata 
INFO[09-11|12:19:43.258] [p2p] GoodPeers                          eth67=0
INFO[09-11|12:19:56.711] [snapshots] Verifying                    progress=3.44%
INFO[09-11|12:20:16.711] [snapshots] Verifying                    progress=7.33%
INFO[09-11|12:20:36.705] [snapshots] Verifying                    progress=11.42%
INFO[09-11|12:20:56.707] [snapshots] Verifying                    progress=17.41%
INFO[09-11|12:21:16.988] [snapshots] Verifying                    progress=21.26%
INFO[09-11|12:21:36.711] [snapshots] Verifying                    progress=25.10%
INFO[09-11|12:21:43.311] [p2p] GoodPeers                          eth67=0
INFO[09-11|12:21:56.714] [snapshots] Verifying                    progress=29.02%
INFO[09-11|12:22:16.699] [snapshots] Verifying                    progress=32.91%
INFO[09-11|12:22:36.703] [snapshots] Verifying                    progress=36.82%
INFO[09-11|12:22:56.710] [snapshots] Verifying                    progress=40.78%
INFO[09-11|12:23:16.708] [snapshots] Verifying                    progress=44.71%
INFO[09-11|12:23:36.705] [snapshots] Verifying                    progress=48.63%
INFO[09-11|12:23:43.330] [p2p] GoodPeers                          eth67=0
INFO[09-11|12:23:56.707] [snapshots] Verifying                    progress=52.50%
INFO[09-11|12:24:16.707] [snapshots] Verifying                    progress=56.41%
INFO[09-11|12:24:36.711] [snapshots] Verifying                    progress=60.26%
INFO[09-11|12:24:56.709] [snapshots] Verifying                    progress=64.18%
INFO[09-11|12:25:16.713] [snapshots] Verifying                    progress=68.03%
INFO[09-11|12:25:36.708] [snapshots] Verifying                    progress=71.91%
INFO[09-11|12:25:43.251] [p2p] GoodPeers                          eth67=0
INFO[09-11|12:25:56.705] [snapshots] Verifying                    progress=75.68%
INFO[09-11|12:26:16.697] [snapshots] Verifying                    progress=79.45%
INFO[09-11|12:26:36.698] [snapshots] Verifying                    progress=83.17%
INFO[09-11|12:26:56.699] [snapshots] Verifying                    progress=87.08%
INFO[09-11|12:27:16.702] [snapshots] Verifying                    progress=90.94%
INFO[09-11|12:27:36.701] [snapshots] Verifying                    progress=94.83%
INFO[09-11|12:27:43.249] [p2p] GoodPeers                          eth67=0
INFO[09-11|12:27:56.703] [snapshots] Verifying                    progress=98.50%
INFO[09-11|12:28:09.769] [Snapshots] Stat                         blocks=0k indices=0k alloc=3.3GB sys=6.7GB
EROR[09-11|12:28:10.021] Staged Sync                              err="[1/16 Headers] BuildMissedIndices: read ParentHash: rlp: element is larger than containing list"
INFO[09-11|12:28:10.523] [Snapshots] Fetching torrent files metadata 
INFO[09-11|12:28:30.539] [snapshots] Verifying                    progress=3.63%
INFO[09-11|12:28:50.533] [snapshots] Verifying                    progress=7.64%
INFO[09-11|12:29:10.525] [snapshots] Verifying                    progress=11.72%
INFO[09-11|12:29:30.541] [snapshots] Verifying                    progress=16.08%
INFO[09-11|12:29:43.253] [p2p] GoodPeers                          eth67=0
INFO[09-11|12:29:50.539] [snapshots] Verifying                    progress=19.97%
INFO[09-11|12:30:10.526] [snapshots] Verifying                    progress=23.86%
INFO[09-11|12:30:30.533] [snapshots] Verifying                    progress=27.77%
INFO[09-11|12:30:50.535] [snapshots] Verifying                    progress=31.66%
INFO[09-11|12:31:10.542] [snapshots] Verifying                    progress=35.58%
INFO[09-11|12:31:30.533] [snapshots] Verifying                    progress=39.42%
INFO[09-11|12:31:43.319] [p2p] GoodPeers                          eth67=0
INFO[09-11|12:31:50.527] [snapshots] Verifying                    progress=43.38%
INFO[09-11|12:32:10.527] [snapshots] Verifying                    progress=47.27%
INFO[09-11|12:32:30.536] [snapshots] Verifying                    progress=51.14%
INFO[09-11|12:32:50.541] [snapshots] Verifying                    progress=55.05%
INFO[09-11|12:33:10.526] [snapshots] Verifying                    progress=58.87%
INFO[09-11|12:33:30.540] [snapshots] Verifying                    progress=62.76%
INFO[09-11|12:33:43.319] [p2p] GoodPeers                          eth67=0
INFO[09-11|12:33:50.537] [snapshots] Verifying                    progress=66.61%
INFO[09-11|12:34:10.538] [snapshots] Verifying                    progress=70.49%
INFO[09-11|12:34:30.528] [snapshots] Verifying                    progress=74.28%
INFO[09-11|12:34:50.542] [snapshots] Verifying                    progress=77.93%
INFO[09-11|12:35:10.535] [snapshots] Verifying                    progress=81.74%
INFO[09-11|12:35:30.533] [snapshots] Verifying                    progress=85.50%
INFO[09-11|12:35:43.249] [p2p] GoodPeers                          eth67=0
INFO[09-11|12:35:50.531] [snapshots] Verifying                    progress=89.40%
INFO[09-11|12:36:10.534] [snapshots] Verifying                    progress=93.35%
INFO[09-11|12:36:30.536] [snapshots] Verifying                    progress=97.04%
INFO[09-11|12:36:50.526] [snapshots] Verifying                    progress=99.89%
INFO[09-11|12:36:51.630] [Snapshots] Stat                         blocks=0k indices=0k alloc=2.5GB sys=6.8GB
EROR[09-11|12:36:51.902] Staged Sync                              err="[1/16 Headers] BuildMissedIndices: read ParentHash: rlp: element is larger than containing list"
INFO[09-11|12:36:52.405] [Snapshots] Fetching torrent files metadata 
INFO[09-11|12:37:12.418] [snapshots] Verifying                    progress=3.69%
INFO[09-11|12:37:32.415] [snapshots] Verifying                    progress=7.69%
INFO[09-11|12:37:43.253] [p2p] GoodPeers                          eth67=0
INFO[09-11|12:37:52.425] [snapshots] Verifying                    progress=11.70%
INFO[09-11|12:38:12.408] [snapshots] Verifying                    progress=15.69%
INFO[09-11|12:38:32.430] [snapshots] Verifying                    progress=19.64%
INFO[09-11|12:38:52.426] [snapshots] Verifying                    progress=23.51%
INFO[09-11|12:39:12.415] [snapshots] Verifying                    progress=27.42%
INFO[09-11|12:39:32.417] [snapshots] Verifying                    progress=31.32%
INFO[09-11|12:39:43.260] [p2p] GoodPeers                          eth67=0
INFO[09-11|12:39:52.417] [snapshots] Verifying                    progress=35.17%
INFO[09-11|12:40:12.415] [snapshots] Verifying                    progress=39.04%
INFO[09-11|12:40:32.407] [snapshots] Verifying                    progress=43.01%
INFO[09-11|12:40:52.422] [snapshots] Verifying                    progress=46.89%
INFO[09-11|12:41:12.614] [snapshots] Verifying                    progress=50.81%
INFO[09-11|12:41:32.429] [snapshots] Verifying                    progress=54.69%
INFO[09-11|12:41:43.272] [p2p] GoodPeers                          eth67=0
INFO[09-11|12:41:52.423] [snapshots] Verifying                    progress=58.56%
INFO[09-11|12:42:12.415] [snapshots] Verifying                    progress=62.42%
INFO[09-11|12:42:32.408] [snapshots] Verifying                    progress=66.33%
INFO[09-11|12:42:52.416] [snapshots] Verifying                    progress=70.14%
INFO[09-11|12:43:12.411] [snapshots] Verifying                    progress=74.04%
INFO[09-11|12:43:32.430] [snapshots] Verifying                    progress=77.76%
INFO[09-11|12:43:43.253] [p2p] GoodPeers                          eth67=0
INFO[09-11|12:43:52.406] [snapshots] Verifying                    progress=81.55%
INFO[09-11|12:44:12.414] [snapshots] Verifying                    progress=85.38%
INFO[09-11|12:44:32.412] [snapshots] Verifying                    progress=89.31%
INFO[09-11|12:44:52.410] [snapshots] Verifying                    progress=92.28%
INFO[09-11|12:45:12.419] [snapshots] Verifying                    progress=95.95%
INFO[09-11|12:45:32.408] [snapshots] Verifying                    progress=99.26%
INFO[09-11|12:45:38.762] [Snapshots] Stat                         blocks=0k indices=0k alloc=2.8GB sys=6.8GB
EROR[09-11|12:45:39.124] Staged Sync                              err="[1/16 Headers] BuildMissedIndices: rlp: expected List"
INFO[09-11|12:45:39.627] [Snapshots] Fetching torrent files metadata 
INFO[09-11|12:45:43.279] [p2p] GoodPeers                          eth67=0
INFO[09-11|12:45:59.645] [snapshots] Verifying                    progress=3.73%
INFO[09-11|12:46:19.635] [snapshots] Verifying                    progress=7.69%
INFO[09-11|12:46:39.632] [snapshots] Verifying                    progress=11.73%
INFO[09-11|12:46:59.645] [snapshots] Verifying                    progress=15.72%
INFO[09-11|12:47:19.744] [snapshots] Verifying                    progress=19.66%
INFO[09-11|12:47:39.646] [snapshots] Verifying                    progress=23.53%
INFO[09-11|12:47:43.248] [p2p] GoodPeers                          eth67=0
INFO[09-11|12:47:59.636] [snapshots] Verifying                    progress=27.47%
INFO[09-11|12:48:19.637] [snapshots] Verifying                    progress=31.36%
INFO[09-11|12:48:39.637] [snapshots] Verifying                    progress=35.22%
INFO[09-11|12:48:59.642] [snapshots] Verifying                    progress=39.01%
INFO[09-11|12:49:19.634] [snapshots] Verifying                    progress=42.99%
INFO[09-11|12:49:39.639] [snapshots] Verifying                    progress=46.88%
INFO[09-11|12:49:43.265] [p2p] GoodPeers                          eth67=0
INFO[09-11|12:49:59.647] [snapshots] Verifying                    progress=50.75%
INFO[09-11|12:50:19.652] [snapshots] Verifying                    progress=54.56%
INFO[09-11|12:50:39.642] [snapshots] Verifying                    progress=58.39%
INFO[09-11|12:50:59.918] [snapshots] Verifying                    progress=62.34%
INFO[09-11|12:51:19.652] [snapshots] Verifying                    progress=66.14%
INFO[09-11|12:51:39.651] [snapshots] Verifying                    progress=69.94%
INFO[09-11|12:51:43.267] [p2p] GoodPeers                          eth67=0
INFO[09-11|12:51:59.631] [snapshots] Verifying                    progress=73.78%
INFO[09-11|12:52:19.675] [snapshots] Verifying                    progress=77.42%
INFO[09-11|12:52:39.633] [snapshots] Verifying                    progress=81.24%
INFO[09-11|12:52:59.633] [snapshots] Verifying                    progress=85.02%
INFO[09-11|12:53:19.631] [snapshots] Verifying                    progress=88.93%
INFO[09-11|12:53:39.631] [snapshots] Verifying                    progress=92.88%
INFO[09-11|12:53:43.249] [p2p] GoodPeers                          eth67=0
INFO[09-11|12:53:59.631] [snapshots] Verifying                    progress=96.58%
INFO[09-11|12:54:19.632] [snapshots] Verifying                    progress=99.60%
INFO[09-11|12:54:23.330] [Snapshots] Stat                         blocks=0k indices=0k alloc=3.5GB sys=6.8GB
EROR[09-11|12:54:23.607] Staged Sync                              err="[1/16 Headers] BuildMissedIndices: read ParentHash: rlp: element is larger than containing list"
INFO[09-11|12:54:24.110] [Snapshots] Fetching torrent files metadata 
INFO[09-11|12:54:44.129] [snapshots] Verifying                    progress=3.88%

@quakemmo
Copy link
Author

Anyone? Please halp!

@blxdyx
Copy link
Collaborator

blxdyx commented Sep 18, 2023

Use the latest release in our repo? your erigon version is wrong

@quakemmo
Copy link
Author

Turns out I was using an old version of erigon-bsc. My bad!
Thanks.

@blxdyx blxdyx closed this as completed Oct 20, 2023
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees

@blxdyx blxdyx

Labels
None yet
Projects
None yet
Milestone
No milestone
Development

No branches or pull requests
2 participants
@quakemmo @blxdyx