Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Flaky: Rotation solver e2e fail (Test_SeqRotation_OneSeq_DA_EVM) #65

Open
danwt opened this issue Jan 7, 2025 · 2 comments
Open

Flaky: Rotation solver e2e fail (Test_SeqRotation_OneSeq_DA_EVM) #65

danwt opened this issue Jan 7, 2025 · 2 comments

Comments

@danwt
Copy link

danwt commented Jan 7, 2025

919224e

Duc is reporting a bug:
2025-01-06T01:17:15.229735Z	error	Rotation solver.	{"error": "search for two rollapp headers: search: target not found: l: 243, r: 243, "}
*2025-01-06T01:17:15.229760Z	error	Sending messages from batch to mempool.

        {
          "path_name": "dymension-demo",
          "src_chain_id": "rollappevm_1234-1",
          "dst_chain_id": "dymension_100-1",
          "src_client_id": "07-tendermint-0",
          "dst_client_id": "07-tendermint-0",
          "types": [
            "/ibc.core.client.v1.MsgUpdateClient",
            "/ibc.core.channel.v1.MsgRecvPacket"
          ],
          "error": "build messages: calculate gas:
           query abci: abci query with options res:
            rpc error: code = Unknown desc = rpc error:
            code = Unknown desc = failed to execute message; message index: 0:
            failed to verify header: cant trust new val set: invalid commit -- insufficient voting power: got 0, needed more than 0
            [cosmos/ibc-go/[email protected]/modules/light-clients/07-tendermint/update.go:114] With gas wanted: '18446744073709551615' and gas used: '367252' : unknown request",

        }


 ��������2025-01-06T01:16:56.631070Z	debug	Starting query cycle	{"chain_id": "rollappevm_1234-1", "chain": "rollappevm_1234-1", "first height": 301}
2025-01-06T01:18:07.8562464Z         ��������2025-01-06T01:16:58.581017Z	debug	Starting query cycle	{"chain_id": "dymension_100-1", "chain": "dymension_100-1", "first height": 554}
2025-01-06T01:18:07.8563524Z         �������>2025-01-06T01:16:58.866530Z	info	Val set trust error, fixing.
2025-01-06T01:18:07.8565242Z         ��������2025-01-06T01:16:59.178055Z	error	Rotation solver.	{"error": "search for two rollapp headers: search: target not found: l: 243, r: 243, "}
2025-01-06T01:18:07.8570271Z         �������*2025-01-06T01:16:59.178077Z	error	Sending messages from batch to mempool.	{"path_name": "dymension-demo", "src_chain_id": "rollappevm_1234-1", "dst_chain_id": "dymension_100-1", "src_client_id": "07-tendermint-0", "dst_client_id": "07-tendermint-0", "types": ["/ibc.core.client.v1.MsgUpdateClient", "/ibc.core.channel.v1.MsgRecvPacket"], "error": "build messages: calculate gas: query abci: abci query with options res: rpc error: code = Unknown desc = rpc error: code = Unknown desc = failed to execute message; message index: 0: failed to verify header: cant trust new val set: invalid commit -- insufficient voting power: got 0, needed more than 0 [cosmos/ibc-go/[email protected]/modules/light-clients/07-tendermint/update.go:114] With gas wanted: '18446744073709551615' and gas used: '374101' : unknown request"}
2025-01-06T01:18:07.8574205Z         ��������2025-01-06T01:17:00.582247Z	debug	Starting query cycle	{"chain_id": "dymension_100-1", "chain": "dymension_100-1", "first height": 555}
2025-01-06T01:18:07.8576519Z         ��������2025-01-06T01:17:02.583108Z	debug	Starting query cycle	{"chain_id": "dymension_100-1", "chain": "dymension_100-1", "first height": 556}
2025-01-06T01:18:07.8577903Z         ��������2025-01-06T01:17:04.584068Z	debug	Starting query cycle	{"chain_id": "dymension_100-1", "chain": "dymension_100-1", "first height": 557}
2025-01-06T01:18:07.8579397Z         ��������2025-01-06T01:17:06.584621Z	debug	Starting query cycle	{"chain_id": "dymension_100-1", "chain": "dymension_100-1", "first height": 558}
2025-01-06T01:18:07.8580874Z         ��������2025-01-06T01:17:08.585060Z	debug	Starting query cycle	{"chain_id": "dymension_100-1", "chain": "dymension_100-1", "first height": 559}
2025-01-06T01:18:07.8588173Z         �������S2025-01-06T01:17:08.593341Z	debug	Assembled message: packet	{"path_name": "dymension-demo", "chain_id": "dymension_100-1", "client_id": "07-tendermint-0", "msg": {"type": "recv_packet", "src_port": "transfer", "src_channel": "channel-0", "dst_port": "transfer", "dst_channel": "channel-0", "sequence": 3, "timeout_height": "1-1254", "timeout_timestamp": 1736126744728567178, "data": "eyJhbW91bnQiOiIxMDAwMDAwIiwiZGVub20iOiJ1cmF4IiwicmVjZWl2ZXIiOiJkeW0xdDBmZmRhYXN4Mmx5czl0ZTk4dHRueHRlNGxoNGZrZmozM2ptdWMiLCJzZW5kZXIiOiJldGhtMTVlczJqN2F5eXEzZzhobjdxNHF1cWpxODlqOGtwN3F5YTA4bnQwIn0=", "ack": ""}}
2025-01-06T01:18:07.8594672Z         �������b2025-01-06T01:17:08.593377Z	debug	sendBatchMessages: will relay messages	{"path_name": "dymension-demo", "chain_id": "dymension_100-1", "client_id": "07-tendermint-0", "msg_0": {"type": "recv_packet", "src_port": "transfer", "src_channel": "channel-0", "dst_port": "transfer", "dst_channel": "channel-0", "sequence": 3, "timeout_height": "1-1254", "timeout_timestamp": 1736126744728567178, "data": "eyJhbW91bnQiOiIxMDAwMDAwIiwiZGVub20iOiJ1cmF4IiwicmVjZWl2ZXIiOiJkeW0xdDBmZmRhYXN4Mmx5czl0ZTk4dHRueHRlNGxoNGZrZmozM2ptdWMiLCJzZW5kZXIiOiJldGhtMTVlczJqN2F5eXEzZzhobjdxNHF1cWpxODlqOGtwN3F5YTA4bnQwIn0=", "ack": ""}}
2025-01-06T01:18:07.8598613Z         ��������2025-01-06T01:17:08.593407Z	debug	Sending messages to mempool.	{"provider_type": "cosmos", "types": ["/ibc.core.client.v1.MsgUpdateClient", "/ibc.core.channel.v1.MsgRecvPacket"], "chain": "dymension_100-1"}
2025-01-06T01:18:07.8600371Z         ��������2025-01-06T01:17:10.585411Z	debug	Starting query cycle	{"chain_id": "dymension_100-1", "chain": "dymension_100-1", "first height": 560}
2025-01-06T01:18:07.8601516Z         ��������2025-01-06T01:17:12.587088Z	debug	Starting query cycle	{"chain_id": "dymension_100-1", "chain": "dymension_100-1", "first height": 561}
2025-01-06T01:18:07.8602877Z         ��������2025-01-06T01:17:14.588889Z	debug	Starting query cycle	{"chain_id": "dymension_100-1", "chain": "dymension_100-1", "first height": 562}
2025-01-06T01:18:07.8603829Z         �������>2025-01-06T01:17:14.920963Z	info	Val set trust error, fixing.
2025-01-06T01:18:07.8604948Z         ��������2025-01-06T01:17:15.229735Z	error	Rotation solver.	{"error": "search for two rollapp headers: search: target not found: l: 243, r: 243, "}
2025-01-06T01:18:07.8610507Z         �������*2025-01-06T01:17:15.229760Z	error	Sending messages from batch to mempool.	{"path_name": "dymension-demo", "src_chain_id": "rollappevm_1234-1", "dst_chain_id": "dymension_100-1", "src_client_id": "07-tendermint-0", "dst_client_id": "07-tendermint-0", "types": ["/ibc.core.client.v1.MsgUpdateClient", "/ibc.core.channel.v1.MsgRecvPacket"], "error": "build messages: calculate gas: query abci: abci query with options res: rpc error: code = Unknown desc = rpc error: code = Unknown desc = failed to execute message; message index: 0: failed to verify header: cant trust new val set: invalid commit -- insufficient voting power: got 0, needed more than 0 [cosmos/ibc-go/[email protected]/modules/light-clients/07-tendermint/update.go:114] With gas wanted: '18446744073709551615' and gas used: '367252' : unknown request"}
2025-01-06T01:18:07.8614302Z         ��������2025-01-06T01:17:16.590208Z	debug	Starting query cycle	{"chain_id": "dymension_100-1", "chain": "dymension_100-1", "first height": 563}
2025-01-06T01:18:07.8616018Z         ��������2025-01-06T01:17:18.590584Z	debug	Starting query cycle	{"chain_id": "dymension_100-1", "chain": "dymension_100-1", "first height": 564}
2025-01-06T01:18:07.8617391Z         ��������2025-01-06T01:17:18.641926Z	debug	Starting query cycle	{"chain_id": "rollappevm_1234-1", "chain": "rollappevm_1234-1", "first height": 309}
2025-01-06T01:18:07.8618735Z         ��������2025-01-06T01:17:20.591150Z	debug	Starting query cycle	{"chain_id": "dymension_100-1", "chain": "dymension_100-1", "first height": 565}
2025-01-06T01:18:07.8620279Z         ��������2025-01-06T01:17:22.592132Z	debug	Starting query cycle	{"chain_id": "dymension_100-1", "chain": "dymension_100-1", "first height": 566}
2025-01-06T01:18:07.8621590Z         ��������2025-01-06T01:17:24.593201Z	debug	Starting query cycle	{"chain_id": "dymension_100-1", "chain": "dymension_100-1", "first height": 567}
2025-01-06T01:18:07.8624645Z         �������t2025-01-06T01:17:24.595592Z	error	Giving up on sending packet message after max retries.	{"path_name": "dymension-demo", "chain_id": "dymension_100-1", "client_id": "07-tendermint-0", "event_type": "recv_packet", "sequence": 3, "channel_id": "channel-0", "port_id": "transfer", "counterparty_channel_id": "channel-0", "counterparty_port_id": "transfer", "max_retries": 5}
2025-01-06T01:18:07.8626699Z         ��������2025-01-06T01:17:26.594392Z	debug	Starting query cycle	{"chain_id": "dymension_100-1", "chain": "dymension_100-1", "first height": 568}
@danwt
Copy link
Author

danwt commented Jan 7, 2025

177_e2e-tests _ rollapp-evm (Test_SeqRotation_OneSeq_DA_EVM).txt

@danwt
Copy link
Author

danwt commented Jan 7, 2025

Added some more logs and a better error, will see if it happens again

@danwt danwt changed the title Rotation solver e2e fail Flaky: Rotation solver e2e fail Jan 7, 2025
@danwt danwt changed the title Flaky: Rotation solver e2e fail Flaky: Rotation solver e2e fail (Test_SeqRotation_OneSeq_DA_EVM) Jan 7, 2025
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees
No one assigned
Labels
None yet
Projects
None yet
Milestone
No milestone
Development

No branches or pull requests
1 participant
@danwt