chain_watcher thread panic, "range end index 196 out of range for slice of length 164"

[DaveWK]

Hi,

I am attempting to sync a base-mainnet node (reth backend for both l1 and l2) with magi.

I downloaded the reth chain archive snapshot from base's website, and the reth node starts OK, but when I restart magi, I see in the logs:

Sep 09 19:39:01 start_magi.sh[3711]: [Mon, 9 Sep 2024 19:39:01] INFO: magi at bin/magi.rs:28 Starting Magi. sync mode=full, network=base
Sep 09 19:39:01 start_magi.sh[3711]: [Mon, 9 Sep 2024 19:39:01] INFO: magi::driver at src/driver/mod.rs:88 starting from head: 0xbcf07afe87ff898f2e2782efb034a39cf3b8a039cf43faaf63431745ac84>
Sep 09 19:39:01 start_magi.sh[3711]: [Mon, 9 Sep 2024 19:39:01] INFO: magi::rpc at src/rpc/mod.rs:144 rpc server started at port 9545
Sep 09 19:39:01 start_magi.sh[3711]: thread 'tokio-runtime-worker' panicked at src/l1/chain_watcher.rs:200:58:
Sep 09 19:39:01 start_magi.sh[3711]: range end index 196 out of range for slice of length 164
Sep 09 19:39:01 start_magi.sh[3711]: note: run with `RUST_BACKTRACE=1` environment variable to display a backtrace

and then there is no output after that; it may be the logging (seems like the verbose is too verbose, but INFO level logs are a little lacking) The process stays running, but it has little (under 15 meg) RAM usage and very little CPU, so I am suspicious the watcher thread died but left the process up.

[DaveWK]

Here's the full backtrace:

Sep 09 19:48:35 start_magi.sh[4612]: [Mon, 9 Sep 2024 19:48:35] INFO: magi at bin/magi.rs:28 Starting Magi. sync mode=full, network=base
Sep 09 19:48:35 start_magi.sh[4612]: [Mon, 9 Sep 2024 19:48:35] INFO: magi::driver at src/driver/mod.rs:88 starting from head: 0xbcf07afe87ff898f2e2782efb034a39cf3b8a039cf43faaf63431745ac84>
Sep 09 19:48:35 start_magi.sh[4612]: [Mon, 9 Sep 2024 19:48:35] INFO: magi::rpc at src/rpc/mod.rs:144 rpc server started at port 9545
Sep 09 19:48:35 start_magi.sh[4612]: thread 'tokio-runtime-worker' panicked at src/l1/chain_watcher.rs:200:58:
Sep 09 19:48:35 start_magi.sh[4612]: range end index 196 out of range for slice of length 164
Sep 09 19:48:35 start_magi.sh[4612]: stack backtrace:
Sep 09 19:48:35 start_magi.sh[4612]:    0:     0x560266029765 - <std::sys::backtrace::BacktraceLock::print::DisplayBacktrace as core::fmt::Display>::fmt::h1b9dad2a88e955ff
Sep 09 19:48:35 start_magi.sh[4612]:    1:     0x560266053cfb - core::fmt::write::h4b5a1270214bc4a7
Sep 09 19:48:35 start_magi.sh[4612]:    2:     0x56026602532f - std::io::Write::write_fmt::hd04af345a50c312d
Sep 09 19:48:35 start_magi.sh[4612]:    3:     0x56026602aa61 - std::panicking::default_hook::{{closure}}::h96ab15e9936be7ed
Sep 09 19:48:35 start_magi.sh[4612]:    4:     0x56026602a73c - std::panicking::default_hook::h3cacb9c27561ad33
Sep 09 19:48:35 start_magi.sh[4612]:    5:     0x56026602b131 - std::panicking::rust_panic_with_hook::hfe205f6954b2c97b
Sep 09 19:48:35 start_magi.sh[4612]:    6:     0x56026602af97 - std::panicking::begin_panic_handler::{{closure}}::h6cb44b3a50f28c44
Sep 09 19:48:35 start_magi.sh[4612]:    7:     0x560266029c29 - std::sys::backtrace::__rust_end_short_backtrace::hf1c1f2a92799bb0e
Sep 09 19:48:35 start_magi.sh[4612]:    8:     0x56026602ac24 - rust_begin_unwind
Sep 09 19:48:35 start_magi.sh[4612]:    9:     0x560266051623 - core::panicking::panic_fmt::h3d8fc78294164da7
Sep 09 19:48:35 start_magi.sh[4612]:   10:     0x560266055d37 - core::slice::index::slice_end_index_len_fail::h87b545b7962eada9
Sep 09 19:48:35 start_magi.sh[4612]:   11:     0x5602653b5959 - magi::l1::chain_watcher::start_watcher::{{closure}}::hb1e55c647ba5c713
Sep 09 19:48:35 start_magi.sh[4612]:   12:     0x56026538ffd4 - tokio::runtime::task::core::Core<T,S>::poll::hf0ecbe3d05f8ea5e
Sep 09 19:48:35 start_magi.sh[4612]:   13:     0x56026543f1da - tokio::runtime::task::harness::Harness<T,S>::poll::hd3c7538220e8c5b4
Sep 09 19:48:35 start_magi.sh[4612]:   14:     0x560265dfb0ba - tokio::runtime::scheduler::multi_thread::worker::Context::run_task::habf92535b796626f
Sep 09 19:48:35 start_magi.sh[4612]:   15:     0x560265dfa4f3 - tokio::runtime::scheduler::multi_thread::worker::Context::run::hc006329c320d688c
Sep 09 19:48:35 start_magi.sh[4612]:   16:     0x560265e1029e - tokio::runtime::context::runtime::enter_runtime::hc73974d4e5dd91ab
Sep 09 19:48:35 start_magi.sh[4612]:   17:     0x560265df9a8a - tokio::runtime::scheduler::multi_thread::worker::run::h935473ee9a71bd8c
Sep 09 19:48:35 start_magi.sh[4612]:   18:     0x560265e1bc25 - tokio::runtime::task::core::Core<T,S>::poll::h8423210808e7feaa
Sep 09 19:48:35 start_magi.sh[4612]:   19:     0x560265df6e6f - tokio::runtime::task::harness::Harness<T,S>::poll::h9f168252bbfb162d
Sep 09 19:48:35 start_magi.sh[4612]:   20:     0x560265e13345 - tokio::runtime::blocking::pool::Inner::run::h2b348b681c4fdce4
Sep 09 19:48:35 start_magi.sh[4612]:   21:     0x560265e09891 - std::sys::backtrace::__rust_begin_short_backtrace::he975c09bda84f4b3
Sep 09 19:48:35 start_magi.sh[4612]:   22:     0x560265e0a042 - core::ops::function::FnOnce::call_once{{vtable.shim}}::hb18e709f70b2822d
Sep 09 19:48:35 start_magi.sh[4612]:   23:     0x56026602e58b - std::sys::pal::unix::thread::Thread::new::thread_start::ha8af9c992ef0b208
Sep 09 19:48:35 start_magi.sh[4612]:   24:     0x7f5280e89c02 - start_thread
Sep 09 19:48:35 start_magi.sh[4612]:   25:     0x7f5280f0ec40 - __GI___clone3
Sep 09 19:48:35 start_magi.sh[4612]:   26:                0x0 - <unknown>