Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

[Network] long delay between receiving a block validation request and getting a response #5366

Open
jcnelson opened this issue Oct 23, 2024 · 4 comments
Open

[Network] long delay between receiving a block validation request and getting a response #5366

jcnelson opened this issue Oct 23, 2024 · 4 comments
Assignees
@jcnelson
Labels
performance-optimizations
Milestone
Nakamoto-3.0.x

Comments

@jcnelson
Copy link
Member

The foundation nakamoto testnet miner sees this:

INFO [1729694751.965402] [testnet/stacks-node/src/nakamoto_node/sign_coordinator.rs:346] [miner.ef7cfd2050ac8b78acb983cd879d0dac12faaf30ab4e3ef400180de87b7809a7] SignCoordinator: beginning to watch for block signatures OR posted blocks., threshold: 1020
INFO [1729694756.336104] [testnet/stacks-node/src/nakamoto_node/sign_coordinator.rs:511] [miner.ef7cfd2050ac8b78acb983cd879d0dac12faaf30ab4e3ef400180de87b7809a7] SignCoordinator: Signature Added to block, block_signer_sighash: 62807f1655cd545c79edd1fb65b07b147c083dbe9ed1072b6cd56d51078b4178, signer_pubkey: 03157271cca37f5ebb4147aad30538fb4b33392ec1cc7161de5f815e29d5146041, signer_slot_id: 2, signature: 01bcc792e49d83b4d7896180dc79bbf19f97eace67d43beda226b3c9a3bb21ae62548e88fb9ec46d10953fafd4ecb173bfd70881229474a31f2558ef1e2953ce95, signer_weight: 1, total_weight_signed: 1, stacks_block_hash: 62807f1655cd545c79edd1fb65b07b147c083dbe9ed1072b6cd56d51078b4178, stacks_block_id: 77b4a778a0072faae62164caf40576cea7fb3828c9d897d423bbe8389703133e, server_version: stacks-signer 3.0.0.0.0 (release/3.0.0.0.0:dd1ebe64603f54dae48558a5d82d9bd885e97a01, release build, linux [x86_64])
INFO [1729694756.692839] [testnet/stacks-node/src/nakamoto_node/sign_coordinator.rs:511] [miner.ef7cfd2050ac8b78acb983cd879d0dac12faaf30ab4e3ef400180de87b7809a7] SignCoordinator: Signature Added to block, block_signer_sighash: 62807f1655cd545c79edd1fb65b07b147c083dbe9ed1072b6cd56d51078b4178, signer_pubkey: 03fbbbc6f26cb4105faf48008f8d7ef1900943a96fdbf63a95b94d7f2d87a4e9bb, signer_slot_id: 4, signature: 01b350643ec788493b0bf1770bddf75b4fc02ac1c7ef09537b2bd42c7029928d4b3311421103ca6c2c55442dd4c1052013ee4c3ae06e6baceb08625aa0665ff393, signer_weight: 1, total_weight_signed: 2, stacks_block_hash: 62807f1655cd545c79edd1fb65b07b147c083dbe9ed1072b6cd56d51078b4178, stacks_block_id: 77b4a778a0072faae62164caf40576cea7fb3828c9d897d423bbe8389703133e, server_version: stacks-signer 3.0.0.0.0 (release/3.0.0.0.0:dd1ebe6, release build, linux [x86_64])
INFO [1729694785.760468] [testnet/stacks-node/src/nakamoto_node/sign_coordinator.rs:511] [miner.ef7cfd2050ac8b78acb983cd879d0dac12faaf30ab4e3ef400180de87b7809a7] SignCoordinator: Signature Added to block, block_signer_sighash: 62807f1655cd545c79edd1fb65b07b147c083dbe9ed1072b6cd56d51078b4178, signer_pubkey: 026ddf4e4e144eac1aad4a215641feee86386a1bff3776f981430364d0ae89cd36, signer_slot_id: 0, signature: 00bd530db16aef6ee107a3d04a471b787618d958191fcc8d8e13ff59f7c769a3aa74b5865fe630ec1dad8bbcc446c0a211606f14d95e3e3fbb82f18ca3b852cb64, signer_weight: 485, total_weight_signed: 487, stacks_block_hash: 62807f1655cd545c79edd1fb65b07b147c083dbe9ed1072b6cd56d51078b4178, stacks_block_id: 77b4a778a0072faae62164caf40576cea7fb3828c9d897d423bbe8389703133e, server_version: stacks-signer 3.0.0.0.0 (release/3.0.0.0.0:dd1ebe6, release build, linux [x86_64])
INFO [1729694788.199587] [testnet/stacks-node/src/nakamoto_node/sign_coordinator.rs:511] [miner.ef7cfd2050ac8b78acb983cd879d0dac12faaf30ab4e3ef400180de87b7809a7] SignCoordinator: Signature Added to block, block_signer_sighash: 62807f1655cd545c79edd1fb65b07b147c083dbe9ed1072b6cd56d51078b4178, signer_pubkey: 02e77164ae21938de2058eb349353f0ae2ac3b76e1a69a03ea94c6fc352164c467, signer_slot_id: 1, signature: 008b9f244f4f92588bdde7a252779b6bf08c4f0fdfa54c19743b2805ee494d40e10bb3d7c93e17ac9a52c23bd37979f55cf2274b289be47912f9e9b73d2fc8c29e, signer_weight: 485, total_weight_signed: 972, stacks_block_hash: 62807f1655cd545c79edd1fb65b07b147c083dbe9ed1072b6cd56d51078b4178, stacks_block_id: 77b4a778a0072faae62164caf40576cea7fb3828c9d897d423bbe8389703133e, server_version: stacks-signer 3.0.0.0.0 (release/3.0.0.0.0:dd1ebe6, release build, linux [x86_64])
INFO [1729694788.199752] [testnet/stacks-node/src/nakamoto_node/sign_coordinator.rs:511] [miner.ef7cfd2050ac8b78acb983cd879d0dac12faaf30ab4e3ef400180de87b7809a7] SignCoordinator: Signature Added to block, block_signer_sighash: 62807f1655cd545c79edd1fb65b07b147c083dbe9ed1072b6cd56d51078b4178, signer_pubkey: 0343b909be3379621d5e68a883e6c74c2222666f4bf6cc38fade2c9d1c57d92213, signer_slot_id: 3, signature: 010a760eb984e3388b3ba03b70a7923cba7e3ded06b4a1de71d7a1bfed723c63c51e3de0f6359b1f9803a91773221d764df160e892d81b5ecfd656f3964384516a, signer_weight: 485, total_weight_signed: 1457, stacks_block_hash: 62807f1655cd545c79edd1fb65b07b147c083dbe9ed1072b6cd56d51078b4178, stacks_block_id: 77b4a778a0072faae62164caf40576cea7fb3828c9d897d423bbe8389703133e, server_version: stacks-signer 3.0.0.0.0 (release/3.0.0.0.0:dd1ebe6, release build, linux [x86_64])
INFO [1729694788.199787] [testnet/stacks-node/src/nakamoto_node/sign_coordinator.rs:594] [miner.ef7cfd2050ac8b78acb983cd879d0dac12faaf30ab4e3ef400180de87b7809a7] SignCoordinator: Received enough signatures. Continuing., stacks_block_hash: 62807f1655cd545c79edd1fb65b07b147c083dbe9ed1072b6cd56d51078b4178, stacks_block_id: 77b4a778a0072faae62164caf40576cea7fb3828c9d897d423bbe8389703133e
INFO [1729694788.569665] [testnet/stacks-node/src/nakamoto_node/miner.rs:404] [miner.ef7cfd2050ac8b78acb983cd879d0dac12faaf30ab4e3ef400180de87b7809a7] Miner: Block signed by signer set and broadcasted, signer_sighash: 62807f1655cd545c79edd1fb65b07b147c083dbe9ed1072b6cd56d51078b4178, stacks_block_hash: 62807f1655cd545c79edd1fb65b07b147c083dbe9ed1072b6cd56d51078b4178, stacks_block_id: 77b4a778a0072faae62164caf40576cea7fb3828c9d897d423bbe8389703133e, block_height: 38268, consensus_hash: d3f81e962f1c9bbc1fe2fe3d434d2b19af0010e0

The hiro nodes and signers see this:

10/23/2024, 02:53:09.682 PM	testnet-seed-nakamoto-0 - Handle incoming Nakamoto block d3f81e962f1c9bbc1fe2fe3d434d2b19af0010e0/62807f1655cd545c79edd1fb65b07b147c083dbe9ed1072b6cd56d51078b4178 obtained via Downloaded
10/23/2024, 02:46:56.231 PM	testnet-stacks-blockchain-nakamoto-3-0 - Relayer: Submitted block-commit
10/23/2024, 02:46:35.321 PM	testnet-stacks-blockchain-nakamoto-3-0 - Handle incoming Nakamoto block d3f81e962f1c9bbc1fe2fe3d434d2b19af0010e0/62807f1655cd545c79edd1fb65b07b147c083dbe9ed1072b6cd56d51078b4178 obtained via Uploaded
10/23/2024, 02:46:34.463 PM	testnet-stacks-blockchain-nakamoto-3-0 - Handle incoming Nakamoto block d3f81e962f1c9bbc1fe2fe3d434d2b19af0010e0/62807f1655cd545c79edd1fb65b07b147c083dbe9ed1072b6cd56d51078b4178 obtained via Uploaded
10/23/2024, 02:46:34.428 PM	testnet-stacks-blockchain-nakamoto-3-0 - Handle incoming Nakamoto block d3f81e962f1c9bbc1fe2fe3d434d2b19af0010e0/62807f1655cd545c79edd1fb65b07b147c083dbe9ed1072b6cd56d51078b4178 obtained via Uploaded
10/23/2024, 02:46:33.235 PM	testnet-seed-nakamoto-0 - Advanced to new tip! d3f81e962f1c9bbc1fe2fe3d434d2b19af0010e0/62807f1655cd545c79edd1fb65b07b147c083dbe9ed1072b6cd56d51078b4178
10/23/2024, 02:46:33.128 PM	testnet-pool-nakamoto-0-0 - Handle incoming Nakamoto block d3f81e962f1c9bbc1fe2fe3d434d2b19af0010e0/62807f1655cd545c79edd1fb65b07b147c083dbe9ed1072b6cd56d51078b4178 obtained via Pushed
10/23/2024, 02:46:31.899 PM	testnet-stacks-blockchain-nakamoto-3-0 - Cycle #4 Signer #1: Got block pushed message
10/23/2024, 02:46:31.846 PM	testnet-stacks-blockchain-nakamoto-3-0 - Cycle #4 Signer #0: Got block pushed message
10/23/2024, 02:46:31.814 PM	testnet-seed-nakamoto-0 - Handle incoming Nakamoto block d3f81e962f1c9bbc1fe2fe3d434d2b19af0010e0/62807f1655cd545c79edd1fb65b07b147c083dbe9ed1072b6cd56d51078b4178 obtained via Pushed
10/23/2024, 02:46:31.784 PM	testnet-stacks-blockchain-nakamoto-3-0 - Cycle #4 Signer #3: Got block pushed message
10/23/2024, 02:46:31.631 PM	testnet-stacks-blockchain-nakamoto-3-0 - Handle incoming Nakamoto block d3f81e962f1c9bbc1fe2fe3d434d2b19af0010e0/62807f1655cd545c79edd1fb65b07b147c083dbe9ed1072b6cd56d51078b4178 obtained via Pushed
10/23/2024, 02:46:31.569 PM	testnet-stacks-blockchain-nakamoto-3-0 - Handle incoming Nakamoto block d3f81e962f1c9bbc1fe2fe3d434d2b19af0010e0/62807f1655cd545c79edd1fb65b07b147c083dbe9ed1072b6cd56d51078b4178 obtained via Pushed
10/23/2024, 02:46:31.511 PM	testnet-stacks-blockchain-nakamoto-3-0 - Handle incoming Nakamoto block d3f81e962f1c9bbc1fe2fe3d434d2b19af0010e0/62807f1655cd545c79edd1fb65b07b147c083dbe9ed1072b6cd56d51078b4178 obtained via Pushed
10/23/2024, 02:46:31.457 PM	testnet-stacks-blockchain-nakamoto-3-0 - Handle incoming Nakamoto block d3f81e962f1c9bbc1fe2fe3d434d2b19af0010e0/62807f1655cd545c79edd1fb65b07b147c083dbe9ed1072b6cd56d51078b4178 obtained via Pushed
10/23/2024, 02:46:30.924 PM	testnet-pool-nakamoto-0-0 - Advanced to new tip! d3f81e962f1c9bbc1fe2fe3d434d2b19af0010e0/62807f1655cd545c79edd1fb65b07b147c083dbe9ed1072b6cd56d51078b4178
10/23/2024, 02:46:30.825 PM	testnet-pool-nakamoto-0-0 - Handle incoming Nakamoto block d3f81e962f1c9bbc1fe2fe3d434d2b19af0010e0/62807f1655cd545c79edd1fb65b07b147c083dbe9ed1072b6cd56d51078b4178 obtained via Pushed
10/23/2024, 02:46:30.473 PM	testnet-seed-nakamoto-0 - Handle incoming Nakamoto block d3f81e962f1c9bbc1fe2fe3d434d2b19af0010e0/62807f1655cd545c79edd1fb65b07b147c083dbe9ed1072b6cd56d51078b4178 obtained via Pushed
10/23/2024, 02:46:29.966 PM	testnet-seed-nakamoto-0 - Handle incoming Nakamoto block d3f81e962f1c9bbc1fe2fe3d434d2b19af0010e0/62807f1655cd545c79edd1fb65b07b147c083dbe9ed1072b6cd56d51078b4178 obtained via Pushed
10/23/2024, 02:46:29.906 PM	testnet-seed-nakamoto-0 - Handle incoming Nakamoto block d3f81e962f1c9bbc1fe2fe3d434d2b19af0010e0/62807f1655cd545c79edd1fb65b07b147c083dbe9ed1072b6cd56d51078b4178 obtained via Pushed
10/23/2024, 02:46:29.821 PM	testnet-seed-nakamoto-0 - Stored incoming Nakamoto block d3f81e962f1c9bbc1fe2fe3d434d2b19af0010e0/62807f1655cd545c79edd1fb65b07b147c083dbe9ed1072b6cd56d51078b4178
10/23/2024, 02:46:29.602 PM	testnet-seed-nakamoto-0 - Handle incoming Nakamoto block d3f81e962f1c9bbc1fe2fe3d434d2b19af0010e0/62807f1655cd545c79edd1fb65b07b147c083dbe9ed1072b6cd56d51078b4178 obtained via Pushed
10/23/2024, 02:46:29.266 PM	testnet-pool-nakamoto-0-0 - Handle incoming Nakamoto block d3f81e962f1c9bbc1fe2fe3d434d2b19af0010e0/62807f1655cd545c79edd1fb65b07b147c083dbe9ed1072b6cd56d51078b4178 obtained via Pushed
10/23/2024, 02:46:29.257 PM	testnet-stacks-blockchain-nakamoto-3-0 - Advanced to new tip! d3f81e962f1c9bbc1fe2fe3d434d2b19af0010e0/62807f1655cd545c79edd1fb65b07b147c083dbe9ed1072b6cd56d51078b4178
10/23/2024, 02:46:29.209 PM	testnet-pool-nakamoto-0-0 - Stored incoming Nakamoto block d3f81e962f1c9bbc1fe2fe3d434d2b19af0010e0/62807f1655cd545c79edd1fb65b07b147c083dbe9ed1072b6cd56d51078b4178
10/23/2024, 02:46:29.036 PM	testnet-pool-nakamoto-0-0 - Handle incoming Nakamoto block d3f81e962f1c9bbc1fe2fe3d434d2b19af0010e0/62807f1655cd545c79edd1fb65b07b147c083dbe9ed1072b6cd56d51078b4178 obtained via Pushed
10/23/2024, 02:46:27.150 PM	testnet-stacks-blockchain-nakamoto-3-0 - Handle incoming Nakamoto block d3f81e962f1c9bbc1fe2fe3d434d2b19af0010e0/62807f1655cd545c79edd1fb65b07b147c083dbe9ed1072b6cd56d51078b4178 obtained via Uploaded
10/23/2024, 02:46:27.116 PM	testnet-stacks-blockchain-nakamoto-3-0 - Handle incoming Nakamoto block d3f81e962f1c9bbc1fe2fe3d434d2b19af0010e0/62807f1655cd545c79edd1fb65b07b147c083dbe9ed1072b6cd56d51078b4178 obtained via Uploaded
10/23/2024, 02:46:27.079 PM	testnet-stacks-blockchain-nakamoto-3-0 - Stored incoming Nakamoto block d3f81e962f1c9bbc1fe2fe3d434d2b19af0010e0/62807f1655cd545c79edd1fb65b07b147c083dbe9ed1072b6cd56d51078b4178
10/23/2024, 02:46:26.926 PM	testnet-stacks-blockchain-nakamoto-3-0 - Handle incoming Nakamoto block d3f81e962f1c9bbc1fe2fe3d434d2b19af0010e0/62807f1655cd545c79edd1fb65b07b147c083dbe9ed1072b6cd56d51078b4178 obtained via Uploaded
10/23/2024, 02:46:23.627 PM	testnet-stacks-blockchain-nakamoto-3-0 - Cycle #4 Signer #3: Broadcasting a block response to stacks node: Accepted(BlockAccepted { signer_signature_hash: 62807f1655cd545c79edd1fb65b07b147c083dbe9ed1072b6cd56d51078b4178, signature: 010a760eb984e3388b3ba03b70a7923cba7e3ded06b4a1de71d7a1bfed723c63c51e3de0f6359b1f9803a91773221d764df160e892d81b5ecfd656f3964384516a, metadata: SignerMessageMetadata { server_version: "stacks-signer 3.0.0.0.0 (release/3.0.0.0.0:dd1ebe6, release build, linux [x86_64])" } })
10/23/2024, 02:46:23.587 PM	testnet-stacks-blockchain-nakamoto-3-0 - Cycle #4 Signer #1: Broadcasting a block response to stacks node: Accepted(BlockAccepted { signer_signature_hash: 62807f1655cd545c79edd1fb65b07b147c083dbe9ed1072b6cd56d51078b4178, signature: 008b9f244f4f92588bdde7a252779b6bf08c4f0fdfa54c19743b2805ee494d40e10bb3d7c93e17ac9a52c23bd37979f55cf2274b289be47912f9e9b73d2fc8c29e, metadata: SignerMessageMetadata { server_version: "stacks-signer 3.0.0.0.0 (release/3.0.0.0.0:dd1ebe6, release build, linux [x86_64])" } })
10/23/2024, 02:46:23.579 PM	testnet-stacks-blockchain-nakamoto-3-0 - Cycle #4 Signer #0: Broadcasting a block response to stacks node: Accepted(BlockAccepted { signer_signature_hash: 62807f1655cd545c79edd1fb65b07b147c083dbe9ed1072b6cd56d51078b4178, signature: 00bd530db16aef6ee107a3d04a471b787618d958191fcc8d8e13ff59f7c769a3aa74b5865fe630ec1dad8bbcc446c0a211606f14d95e3e3fbb82f18ca3b852cb64, metadata: SignerMessageMetadata { server_version: "stacks-signer 3.0.0.0.0 (release/3.0.0.0.0:dd1ebe6, release build, linux [x86_64])" } })
10/23/2024, 02:46:23.529 PM	testnet-stacks-blockchain-nakamoto-3-0 - Cycle #4 Signer #3: Received a block validate response: Ok(BlockValidateOk { signer_signature_hash: 62807f1655cd545c79edd1fb65b07b147c083dbe9ed1072b6cd56d51078b4178, cost: ExecutionCost { write_length: 2676198, write_count: 1227, read_length: 1099372, read_count: 1207, runtime: 76922159 }, size: 498559 })
10/23/2024, 02:46:23.516 PM	testnet-stacks-blockchain-nakamoto-3-0 - Cycle #4 Signer #1: Received a block validate response: Ok(BlockValidateOk { signer_signature_hash: 62807f1655cd545c79edd1fb65b07b147c083dbe9ed1072b6cd56d51078b4178, cost: ExecutionCost { write_length: 2676198, write_count: 1227, read_length: 1099372, read_count: 1207, runtime: 76922159 }, size: 498559 })
10/23/2024, 02:46:23.502 PM	testnet-stacks-blockchain-nakamoto-3-0 - Cycle #4 Signer #0: Received a block validate response: Ok(BlockValidateOk { signer_signature_hash: 62807f1655cd545c79edd1fb65b07b147c083dbe9ed1072b6cd56d51078b4178, cost: ExecutionCost { write_length: 2676198, write_count: 1227, read_length: 1099372, read_count: 1207, runtime: 76922159 }, size: 498559 })
10/23/2024, 02:46:23.494 PM	testnet-stacks-blockchain-nakamoto-3-0 - Participant: validated anchored block
10/23/2024, 02:46:22.486 PM	testnet-stacks-blockchain-nakamoto-3-0 - Received block proposal request
10/23/2024, 02:46:21.392 PM	testnet-stacks-blockchain-nakamoto-3-0 - Received block proposal request
10/23/2024, 02:46:21.370 PM	testnet-stacks-blockchain-nakamoto-3-0 - Received block proposal request
10/23/2024, 02:45:53.205 PM	testnet-stacks-blockchain-nakamoto-3-0 - Cycle #4 Signer #1: received a block proposal for a new block. Submit block for validation. 
10/23/2024, 02:45:53.141 PM	testnet-stacks-blockchain-nakamoto-3-0 - Cycle #4 Signer #0: received a block proposal for a new block. Submit block for validation. 
10/23/2024, 02:45:53.072 PM	testnet-stacks-blockchain-nakamoto-3-0 - Cycle #4 Signer #3: received a block proposal for a new block. Submit block for validation.

Why is there such a long delay between the signer submitting a block proposal and the node receiving it?
@jcnelson jcnelson self-assigned this Oct 23, 2024
@github-project-automation github-project-automation bot moved this to Status: 🆕 New in Stacks Core Eng Oct 23, 2024
@jcnelson
Copy link
Member Author

Seems the root cause is that these nodes are just super-backlogged with HTTP requests:

10/23/2024, 02:46:23.627 PM	testnet-stacks-blockchain-nakamoto-3-0 - Cycle #4 Signer #3: Broadcasting a block response to stacks node: Accepted(BlockAccepted { signer_signature_hash: 62807f1655cd545c79edd1fb65b07b147c083dbe9ed1072b6cd56d51078b4178, signature: 010a760eb984e3388b3ba03b70a7923cba7e3ded06b4a1de71d7a1bfed723c63c51e3de0f6359b1f9803a91773221d764df160e892d81b5ecfd656f3964384516a, metadata: SignerMessageMetadata { server_version: "stacks-signer 3.0.0.0.0 (release/3.0.0.0.0:dd1ebe6, release build, linux [x86_64])" } }) -  - 
10/23/2024, 02:46:23.604 PM	testnet-stacks-blockchain-nakamoto-3-0 - Handled StacksHTTPRequest - /v2/transactions - 156
10/23/2024, 02:46:23.587 PM	testnet-stacks-blockchain-nakamoto-3-0 - Cycle #4 Signer #1: Broadcasting a block response to stacks node: Accepted(BlockAccepted { signer_signature_hash: 62807f1655cd545c79edd1fb65b07b147c083dbe9ed1072b6cd56d51078b4178, signature: 008b9f244f4f92588bdde7a252779b6bf08c4f0fdfa54c19743b2805ee494d40e10bb3d7c93e17ac9a52c23bd37979f55cf2274b289be47912f9e9b73d2fc8c29e, metadata: SignerMessageMetadata { server_version: "stacks-signer 3.0.0.0.0 (release/3.0.0.0.0:dd1ebe6, release build, linux [x86_64])" } }) -  - 
10/23/2024, 02:46:23.579 PM	testnet-stacks-blockchain-nakamoto-3-0 - Cycle #4 Signer #0: Broadcasting a block response to stacks node: Accepted(BlockAccepted { signer_signature_hash: 62807f1655cd545c79edd1fb65b07b147c083dbe9ed1072b6cd56d51078b4178, signature: 00bd530db16aef6ee107a3d04a471b787618d958191fcc8d8e13ff59f7c769a3aa74b5865fe630ec1dad8bbcc446c0a211606f14d95e3e3fbb82f18ca3b852cb64, metadata: SignerMessageMetadata { server_version: "stacks-signer 3.0.0.0.0 (release/3.0.0.0.0:dd1ebe6, release build, linux [x86_64])" } }) -  - 
10/23/2024, 02:46:23.529 PM	testnet-stacks-blockchain-nakamoto-3-0 - Cycle #4 Signer #3: Received a block validate response: Ok(BlockValidateOk { signer_signature_hash: 62807f1655cd545c79edd1fb65b07b147c083dbe9ed1072b6cd56d51078b4178, cost: ExecutionCost { write_length: 2676198, write_count: 1227, read_length: 1099372, read_count: 1207, runtime: 76922159 }, size: 498559 }) -  - 
10/23/2024, 02:46:23.516 PM	testnet-stacks-blockchain-nakamoto-3-0 - Cycle #4 Signer #1: Received a block validate response: Ok(BlockValidateOk { signer_signature_hash: 62807f1655cd545c79edd1fb65b07b147c083dbe9ed1072b6cd56d51078b4178, cost: ExecutionCost { write_length: 2676198, write_count: 1227, read_length: 1099372, read_count: 1207, runtime: 76922159 }, size: 498559 }) -  - 
10/23/2024, 02:46:23.502 PM	testnet-stacks-blockchain-nakamoto-3-0 - Cycle #4 Signer #0: Received a block validate response: Ok(BlockValidateOk { signer_signature_hash: 62807f1655cd545c79edd1fb65b07b147c083dbe9ed1072b6cd56d51078b4178, cost: ExecutionCost { write_length: 2676198, write_count: 1227, read_length: 1099372, read_count: 1207, runtime: 76922159 }, size: 498559 }) -  - 
10/23/2024, 02:46:23.494 PM	testnet-stacks-blockchain-nakamoto-3-0 - Participant: validated anchored block -  - 
10/23/2024, 02:46:23.447 PM	testnet-stacks-blockchain-nakamoto-3-0 - Handled StacksHTTPRequest - /v2/transactions - 73
10/23/2024, 02:46:23.372 PM	testnet-stacks-blockchain-nakamoto-3-0 - Handled StacksHTTPRequest - /v2/transactions - 145
10/23/2024, 02:46:22.814 PM	testnet-stacks-blockchain-nakamoto-3-0 - Handled StacksHTTPRequest - /v2/transactions - 130
10/23/2024, 02:46:22.684 PM	testnet-stacks-blockchain-nakamoto-3-0 - Handled StacksHTTPRequest - /v2/pox - 122
10/23/2024, 02:46:22.560 PM	testnet-stacks-blockchain-nakamoto-3-0 - Handled StacksHTTPRequest - /v2/transactions - 72
10/23/2024, 02:46:22.486 PM	testnet-stacks-blockchain-nakamoto-3-0 - Handled StacksHTTPRequest - /v3/block_proposal - 0
10/23/2024, 02:46:22.486 PM	testnet-stacks-blockchain-nakamoto-3-0 - Received block proposal request -  - 
10/23/2024, 02:46:22.111 PM	testnet-stacks-blockchain-nakamoto-3-0 - Handled StacksHTTPRequest - /v2/transactions - 110
10/23/2024, 02:46:22.000 PM	testnet-stacks-blockchain-nakamoto-3-0 - Handled StacksHTTPRequest - /v2/transactions - 0
10/23/2024, 02:46:22.000 PM	testnet-stacks-blockchain-nakamoto-3-0 - Handled StacksHTTPRequest - /v2/transactions - 150
10/23/2024, 02:46:21.536 PM	testnet-stacks-blockchain-nakamoto-3-0 - Handled StacksHTTPRequest - /v2/accounts/ST2D1M978SCE52GAV07VXSRC9DQBP69X5WHX0DHN5?proof=0 - 46
10/23/2024, 02:46:21.489 PM	testnet-stacks-blockchain-nakamoto-3-0 - Handled StacksHTTPRequest - /v2/accounts/ST484MS3VACPAZ90WHC21XQ7T6XANCV341HJYE0W?proof=0 - 47
10/23/2024, 02:46:21.441 PM	testnet-stacks-blockchain-nakamoto-3-0 - Handled StacksHTTPRequest - /v2/accounts/ST3MNK12DGQF7JN4Q0STK6926VWE5MN21KJ4EGV0E?proof=0 - 48
10/23/2024, 02:46:21.392 PM	testnet-stacks-blockchain-nakamoto-3-0 - Handled StacksHTTPRequest - /v3/block_proposal - 0
10/23/2024, 02:46:21.392 PM	testnet-stacks-blockchain-nakamoto-3-0 - Received block proposal request -  - 
10/23/2024, 02:46:21.373 PM	testnet-stacks-blockchain-nakamoto-3-0 - Handled StacksHTTPRequest - /v2/info - 0
10/23/2024, 02:46:21.373 PM	testnet-stacks-blockchain-nakamoto-3-0 - Handled StacksHTTPRequest - /v3/block_proposal - 2
10/23/2024, 02:46:21.370 PM	testnet-stacks-blockchain-nakamoto-3-0 - Received block proposal request -  - 
10/23/2024, 02:46:20.965 PM	testnet-stacks-blockchain-nakamoto-3-0 - Handled StacksHTTPRequest - /v2/transactions - 133
10/23/2024, 02:46:20.831 PM	testnet-stacks-blockchain-nakamoto-3-0 - Handled StacksHTTPRequest - /v3/tenures/info - 0
10/23/2024, 02:46:20.830 PM	testnet-stacks-blockchain-nakamoto-3-0 - Handled StacksHTTPRequest - /v2/transactions - 129
10/23/2024, 02:46:20.700 PM	testnet-stacks-blockchain-nakamoto-3-0 - Handled StacksHTTPRequest - /v2/transactions - 136
10/23/2024, 02:46:20.255 PM	testnet-stacks-blockchain-nakamoto-3-0 - Handled StacksHTTPRequest - /v2/pox - 117
10/23/2024, 02:46:20.137 PM	testnet-stacks-blockchain-nakamoto-3-0 - Handled StacksHTTPRequest - /v2/transactions - 108
10/23/2024, 02:46:19.750 PM	testnet-stacks-blockchain-nakamoto-3-0 - Handled StacksHTTPRequest - /v2/transactions - 138
10/23/2024, 02:46:19.224 PM	testnet-stacks-blockchain-nakamoto-3-0 - Handled StacksHTTPRequest - /v2/transactions - 134
10/23/2024, 02:46:19.090 PM	testnet-stacks-blockchain-nakamoto-3-0 - Handled StacksHTTPRequest - /v2/transactions - 0
10/23/2024, 02:46:19.089 PM	testnet-stacks-blockchain-nakamoto-3-0 - Handled StacksHTTPRequest - /v2/transactions - 68
10/23/2024, 02:46:19.020 PM	testnet-stacks-blockchain-nakamoto-3-0 - Handled StacksHTTPRequest - /v2/transactions - 69
10/23/2024, 02:46:18.593 PM	testnet-stacks-blockchain-nakamoto-3-0 - Handled StacksHTTPRequest - /v2/transactions - 106
10/23/2024, 02:46:18.486 PM	testnet-stacks-blockchain-nakamoto-3-0 - Handled StacksHTTPRequest - /v2/transactions - 65
10/23/2024, 02:46:18.420 PM	testnet-stacks-blockchain-nakamoto-3-0 - Handled StacksHTTPRequest - /v2/accounts/ST2D1M978SCE52GAV07VXSRC9DQBP69X5WHX0DHN5?proof=0 - 47
10/23/2024, 02:46:18.372 PM	testnet-stacks-blockchain-nakamoto-3-0 - Handled StacksHTTPRequest - /v2/accounts/ST484MS3VACPAZ90WHC21XQ7T6XANCV341HJYE0W?proof=0 - 48
10/23/2024, 02:46:18.323 PM	testnet-stacks-blockchain-nakamoto-3-0 - Handled StacksHTTPRequest - /v2/accounts/ST3MNK12DGQF7JN4Q0STK6926VWE5MN21KJ4EGV0E?proof=0 - 48
10/23/2024, 02:46:18.271 PM	testnet-stacks-blockchain-nakamoto-3-0 - Handled StacksHTTPRequest - /v2/transactions - 75
10/23/2024, 02:46:17.901 PM	testnet-stacks-blockchain-nakamoto-3-0 - Handled StacksHTTPRequest - /v2/transactions - 65
10/23/2024, 02:46:17.834 PM	testnet-stacks-blockchain-nakamoto-3-0 - Handled StacksHTTPRequest - /v2/pox - 116
10/23/2024, 02:46:17.717 PM	testnet-stacks-blockchain-nakamoto-3-0 - Handled StacksHTTPRequest - /v2/transactions - 66
10/23/2024, 02:46:17.650 PM	testnet-stacks-blockchain-nakamoto-3-0 - Handled StacksHTTPRequest - /v2/transactions - 103
10/23/2024, 02:46:17.546 PM	testnet-stacks-blockchain-nakamoto-3-0 - Handled StacksHTTPRequest - /v2/transactions - 128
10/23/2024, 02:46:17.016 PM	testnet-stacks-blockchain-nakamoto-3-0 - Handled StacksHTTPRequest - /v2/transactions - 115
10/23/2024, 02:46:16.900 PM	testnet-stacks-blockchain-nakamoto-3-0 - Handled StacksHTTPRequest - /v2/transactions - 142
10/23/2024, 02:46:16.758 PM	testnet-stacks-blockchain-nakamoto-3-0 - Handled StacksHTTPRequest - /v2/transactions - 128
10/23/2024, 02:46:16.629 PM	testnet-stacks-blockchain-nakamoto-3-0 - Handled StacksHTTPRequest - /v2/transactions - 107
10/23/2024, 02:46:16.520 PM	testnet-stacks-blockchain-nakamoto-3-0 - Handled StacksHTTPRequest - /v2/transactions - 131
10/23/2024, 02:46:16.045 PM	testnet-stacks-blockchain-nakamoto-3-0 - Handled StacksHTTPRequest - /v2/info - 0
10/23/2024, 02:46:16.045 PM	testnet-stacks-blockchain-nakamoto-3-0 - Handled StacksHTTPRequest - /v3/tenures/info - 0
10/23/2024, 02:46:16.044 PM	testnet-stacks-blockchain-nakamoto-3-0 - Handled StacksHTTPRequest - /v2/transactions - 130
10/23/2024, 02:46:15.913 PM	testnet-stacks-blockchain-nakamoto-3-0 - Handled StacksHTTPRequest - /v2/transactions - 130
10/23/2024, 02:46:15.782 PM	testnet-stacks-blockchain-nakamoto-3-0 - Handled StacksHTTPRequest - /v2/accounts/ST1114TBQYGNPGFAVXKWBKZAHP0X7ZGX9K6XYYE4F?proof=0 - 51
10/23/2024, 02:46:15.729 PM	testnet-stacks-blockchain-nakamoto-3-0 - Handled StacksHTTPRequest - /v2/accounts/ST2Q6124HQFKVKPJSS5J6156BJR74FD6EC1297HJ1?proof=0 - 47
10/23/2024, 02:46:15.681 PM	testnet-stacks-blockchain-nakamoto-3-0 - Handled StacksHTTPRequest - /v2/transactions - 114
10/23/2024, 02:46:15.566 PM	testnet-stacks-blockchain-nakamoto-3-0 - Handled StacksHTTPRequest - /v2/accounts/ST20Q2N56E1NBWE37R4VGSF89X4HHTB3GSMD8GKYW?proof=0 - 47
10/23/2024, 02:46:15.518 PM	testnet-stacks-blockchain-nakamoto-3-0 - Handled StacksHTTPRequest - /v2/transactions - 69
10/23/2024, 02:46:15.448 PM	testnet-stacks-blockchain-nakamoto-3-0 - Handled StacksHTTPRequest - /v2/transactions - 0
10/23/2024, 02:46:15.446 PM	testnet-stacks-blockchain-nakamoto-3-0 - Handled StacksHTTPRequest - /v2/transactions - 71
10/23/2024, 02:46:15.025 PM	testnet-stacks-blockchain-nakamoto-3-0 - Handled StacksHTTPRequest - /v2/contracts/interface/ST000000000000000000002AMW42H/pox-4 - 48
10/23/2024, 02:46:14.976 PM	testnet-stacks-blockchain-nakamoto-3-0 - Handled StacksHTTPRequest - /v2/contracts/interface/ST000000000000000000002AMW42H/pox-4 - 51
10/23/2024, 02:46:14.924 PM	testnet-stacks-blockchain-nakamoto-3-0 - Handled StacksHTTPRequest - /v2/pox - 115
10/23/2024, 02:46:14.808 PM	testnet-stacks-blockchain-nakamoto-3-0 - Handled StacksHTTPRequest - /v2/transactions - 102
10/23/2024, 02:46:14.705 PM	testnet-stacks-blockchain-nakamoto-3-0 - Handled StacksHTTPRequest - /v2/contracts/interface/ST000000000000000000002AMW42H/pox-4 - 47
10/23/2024, 02:46:14.657 PM	testnet-stacks-blockchain-nakamoto-3-0 - Handled StacksHTTPRequest - /v2/transactions - 129
10/23/2024, 02:46:14.527 PM	testnet-stacks-blockchain-nakamoto-3-0 - Handled StacksHTTPRequest - /v2/accounts/ST2D1M978SCE52GAV07VXSRC9DQBP69X5WHX0DHN5?proof=0 - 46
10/23/2024, 02:46:14.479 PM	testnet-stacks-blockchain-nakamoto-3-0 - Handled StacksHTTPRequest - /v2/accounts/ST484MS3VACPAZ90WHC21XQ7T6XANCV341HJYE0W?proof=0 - 46
10/23/2024, 02:46:14.432 PM	testnet-stacks-blockchain-nakamoto-3-0 - Handled StacksHTTPRequest - /v2/accounts/ST3MNK12DGQF7JN4Q0STK6926VWE5MN21KJ4EGV0E?proof=0 - 47
10/23/2024, 02:46:14.047 PM	testnet-stacks-blockchain-nakamoto-3-0 - Handled StacksHTTPRequest - /v2/pox - 120
10/23/2024, 02:46:13.927 PM	testnet-stacks-blockchain-nakamoto-3-0 - Handled StacksHTTPRequest - /v2/pox - 126
10/23/2024, 02:46:13.799 PM	testnet-stacks-blockchain-nakamoto-3-0 - Handled StacksHTTPRequest - /v2/pox - 127
10/23/2024, 02:46:13.671 PM	testnet-stacks-blockchain-nakamoto-3-0 - Handled StacksHTTPRequest - /v2/transactions - 71
10/23/2024, 02:46:13.599 PM	testnet-stacks-blockchain-nakamoto-3-0 - Handled StacksHTTPRequest - /v2/transactions - 135
10/23/2024, 02:46:13.074 PM	testnet-stacks-blockchain-nakamoto-3-0 - Handled StacksHTTPRequest - /v2/transactions - 103
10/23/2024, 02:46:12.970 PM	testnet-stacks-blockchain-nakamoto-3-0 - Handled StacksHTTPRequest - /v2/transactions - 148
10/23/2024, 02:46:12.817 PM	testnet-stacks-blockchain-nakamoto-3-0 - Handled StacksHTTPRequest - /v2/accounts/ST1114TBQYGNPGFAVXKWBKZAHP0X7ZGX9K6XYYE4F?proof=0 - 53
10/23/2024, 02:46:12.763 PM	testnet-stacks-blockchain-nakamoto-3-0 - Handled StacksHTTPRequest - /v2/accounts/ST2Q6124HQFKVKPJSS5J6156BJR74FD6EC1297HJ1?proof=0 - 54
10/23/2024, 02:46:12.328 PM	testnet-stacks-blockchain-nakamoto-3-0 - Handled StacksHTTPRequest - /v2/transactions - 100
10/23/2024, 02:46:12.226 PM	testnet-stacks-blockchain-nakamoto-3-0 - Handled StacksHTTPRequest - /v2/transactions - 64
10/23/2024, 02:46:12.161 PM	testnet-stacks-blockchain-nakamoto-3-0 - Handled StacksHTTPRequest - /v2/transactions - 127
10/23/2024, 02:46:12.034 PM	testnet-stacks-blockchain-nakamoto-3-0 - Handled StacksHTTPRequest - /v2/transactions - 69
10/23/2024, 02:46:11.963 PM	testnet-stacks-blockchain-nakamoto-3-0 - Handled StacksHTTPRequest - /v2/transactions - 113
10/23/2024, 02:46:11.849 PM	testnet-stacks-blockchain-nakamoto-3-0 - Handled StacksHTTPRequest - /v2/pox - 127
10/23/2024, 02:46:11.721 PM	testnet-stacks-blockchain-nakamoto-3-0 - Handled StacksHTTPRequest - /v2/transactions - 0
10/23/2024, 02:46:11.720 PM	testnet-stacks-blockchain-nakamoto-3-0 - Handled StacksHTTPRequest - /v2/accounts/ST20Q2N56E1NBWE37R4VGSF89X4HHTB3GSMD8GKYW?proof=0 - 49
10/23/2024, 02:46:11.173 PM	testnet-stacks-blockchain-nakamoto-3-0 - Handled StacksHTTPRequest - /v2/transactions - 67
10/23/2024, 02:46:11.105 PM	testnet-stacks-blockchain-nakamoto-3-0 - Handled StacksHTTPRequest - /v3/tenures/info - 0
10/23/2024, 02:46:11.104 PM	testnet-stacks-blockchain-nakamoto-3-0 - Handled StacksHTTPRequest - /v2/accounts/ST484MS3VACPAZ90WHC21XQ7T6XANCV341HJYE0W?proof=0 - 48
10/23/2024, 02:46:11.055 PM	testnet-stacks-blockchain-nakamoto-3-0 - Handled StacksHTTPRequest - /v2/accounts/ST2D1M978SCE52GAV07VXSRC9DQBP69X5WHX0DHN5?proof=0 - 47
10/23/2024, 02:46:11.007 PM	testnet-stacks-blockchain-nakamoto-3-0 - Handled StacksHTTPRequest - /v2/accounts/ST3MNK12DGQF7JN4Q0STK6926VWE5MN21KJ4EGV0E?proof=0 - 48
10/23/2024, 02:46:10.958 PM	testnet-stacks-blockchain-nakamoto-3-0 - Handled StacksHTTPRequest - /v2/pox - 119
10/23/2024, 02:46:10.480 PM	testnet-stacks-blockchain-nakamoto-3-0 - Handled StacksHTTPRequest - /v2/transactions - 134
10/23/2024, 02:46:10.344 PM	testnet-stacks-blockchain-nakamoto-3-0 - Handled StacksHTTPRequest - /v2/info - 0
10/23/2024, 02:46:10.343 PM	testnet-stacks-blockchain-nakamoto-3-0 - Handled StacksHTTPRequest - /v2/transactions - 143
10/23/2024, 02:46:10.198 PM	testnet-stacks-blockchain-nakamoto-3-0 - Handled StacksHTTPRequest - /v2/transactions - 69
10/23/2024, 02:46:10.128 PM	testnet-stacks-blockchain-nakamoto-3-0 - Handled StacksHTTPRequest - /v2/transactions - 156
10/23/2024, 02:46:09.971 PM	testnet-stacks-blockchain-nakamoto-3-0 - Handled StacksHTTPRequest - /v2/transactions - 136
10/23/2024, 02:46:09.438 PM	testnet-stacks-blockchain-nakamoto-3-0 - Handled StacksHTTPRequest - /v2/transactions - 135
10/23/2024, 02:46:09.303 PM	testnet-stacks-blockchain-nakamoto-3-0 - Handled StacksHTTPRequest - /v2/transactions - 66
10/23/2024, 02:46:09.235 PM	testnet-stacks-blockchain-nakamoto-3-0 - Handled StacksHTTPRequest - /v2/transactions - 114
10/23/2024, 02:46:09.120 PM	testnet-stacks-blockchain-nakamoto-3-0 - Handled StacksHTTPRequest - /v2/pox - 127
10/23/2024, 02:46:08.991 PM	testnet-stacks-blockchain-nakamoto-3-0 - Handled StacksHTTPRequest - /v2/transactions - 136
10/23/2024, 02:46:08.507 PM	testnet-stacks-blockchain-nakamoto-3-0 - Handled StacksHTTPRequest - /v3/tenures/info - 0
10/23/2024, 02:46:08.507 PM	testnet-stacks-blockchain-nakamoto-3-0 - Handled StacksHTTPRequest - /v2/transactions - 106
10/23/2024, 02:46:08.400 PM	testnet-stacks-blockchain-nakamoto-3-0 - Handled StacksHTTPRequest - /v2/transactions - 137
10/23/2024, 02:46:08.261 PM	testnet-stacks-blockchain-nakamoto-3-0 - Handled StacksHTTPRequest - /v2/transactions - 110
10/23/2024, 02:46:08.150 PM	testnet-stacks-blockchain-nakamoto-3-0 - Handled StacksHTTPRequest - /v2/transactions - 0
10/23/2024, 02:46:07.723 PM	testnet-stacks-blockchain-nakamoto-3-0 - Handled StacksHTTPRequest - /v2/transactions - 67
10/23/2024, 02:46:07.655 PM	testnet-stacks-blockchain-nakamoto-3-0 - Handled StacksHTTPRequest - /v2/transactions - 129
10/23/2024, 02:46:07.523 PM	testnet-stacks-blockchain-nakamoto-3-0 - Handled StacksHTTPRequest - /v2/accounts/ST2D1M978SCE52GAV07VXSRC9DQBP69X5WHX0DHN5?proof=0 - 47
10/23/2024, 02:46:07.476 PM	testnet-stacks-blockchain-nakamoto-3-0 - Handled StacksHTTPRequest - /v2/accounts/ST484MS3VACPAZ90WHC21XQ7T6XANCV341HJYE0W?proof=0 - 47
10/23/2024, 02:46:07.427 PM	testnet-stacks-blockchain-nakamoto-3-0 - Handled StacksHTTPRequest - /v2/accounts/ST3MNK12DGQF7JN4Q0STK6926VWE5MN21KJ4EGV0E?proof=0 - 47
10/23/2024, 02:46:07.075 PM	testnet-stacks-blockchain-nakamoto-3-0 - Handled StacksHTTPRequest - /v2/transactions - 118
10/23/2024, 02:46:06.955 PM	testnet-stacks-blockchain-nakamoto-3-0 - Handled StacksHTTPRequest - /v2/transactions - 112
10/23/2024, 02:46:06.842 PM	testnet-stacks-blockchain-nakamoto-3-0 - Handled StacksHTTPRequest - /v2/transactions - 79
10/23/2024, 02:46:06.762 PM	testnet-stacks-blockchain-nakamoto-3-0 - Handled StacksHTTPRequest - /v2/transactions - 67
10/23/2024, 02:46:06.247 PM	testnet-stacks-blockchain-nakamoto-3-0 - Handled StacksHTTPRequest - /v2/transactions - 73
10/23/2024, 02:46:06.173 PM	testnet-stacks-blockchain-nakamoto-3-0 - Handled StacksHTTPRequest - /v2/pox - 133
10/23/2024, 02:46:06.039 PM	testnet-stacks-blockchain-nakamoto-3-0 - Handled StacksHTTPRequest - /v2/transactions - 133
10/23/2024, 02:46:05.622 PM	testnet-stacks-blockchain-nakamoto-3-0 - Handled StacksHTTPRequest - /v3/tenures/info - 0
10/23/2024, 02:46:05.621 PM	testnet-stacks-blockchain-nakamoto-3-0 - Handled StacksHTTPRequest - /v2/transactions - 126
10/23/2024, 02:46:05.494 PM	testnet-stacks-blockchain-nakamoto-3-0 - Handled StacksHTTPRequest - /v2/transactions - 101
10/23/2024, 02:46:05.286 PM	testnet-stacks-blockchain-nakamoto-3-0 - Handled StacksHTTPRequest - /v3/tenures/0ca0d2f26b6a1d4716884be8f3140cac4752385c5e60314bcb74fca6d38bd788 - 17
10/23/2024, 02:46:04.954 PM	testnet-stacks-blockchain-nakamoto-3-0 - Handled StacksHTTPRequest - /v2/transactions - 130
10/23/2024, 02:46:04.824 PM	testnet-stacks-blockchain-nakamoto-3-0 - Handled StacksHTTPRequest - /v2/transactions - 125
10/23/2024, 02:46:04.697 PM	testnet-stacks-blockchain-nakamoto-3-0 - Handled StacksHTTPRequest - /v2/transactions - 0
10/23/2024, 02:46:04.696 PM	testnet-stacks-blockchain-nakamoto-3-0 - Handled StacksHTTPRequest - /v2/transactions - 67
10/23/2024, 02:46:04.629 PM	testnet-stacks-blockchain-nakamoto-3-0 - Handled StacksHTTPRequest - /v2/info - 0
10/23/2024, 02:46:04.226 PM	testnet-stacks-blockchain-nakamoto-3-0 - Handled StacksHTTPRequest - /v2/transactions - 114
10/23/2024, 02:46:04.111 PM	testnet-stacks-blockchain-nakamoto-3-0 - Handled StacksHTTPRequest - /v2/accounts/ST2D1M978SCE52GAV07VXSRC9DQBP69X5WHX0DHN5?proof=0 - 47
10/23/2024, 02:46:04.063 PM	testnet-stacks-blockchain-nakamoto-3-0 - Handled StacksHTTPRequest - /v2/accounts/ST484MS3VACPAZ90WHC21XQ7T6XANCV341HJYE0W?proof=0 - 65
10/23/2024, 02:46:03.997 PM	testnet-stacks-blockchain-nakamoto-3-0 - Handled StacksHTTPRequest - /v2/accounts/ST3MNK12DGQF7JN4Q0STK6926VWE5MN21KJ4EGV0E?proof=0 - 50
10/23/2024, 02:46:03.621 PM	testnet-stacks-blockchain-nakamoto-3-0 - Handled StacksHTTPRequest - /v2/pox - 118
10/23/2024, 02:46:03.502 PM	testnet-stacks-blockchain-nakamoto-3-0 - Handled StacksHTTPRequest - /v2/transactions - 89
10/23/2024, 02:46:03.147 PM	testnet-stacks-blockchain-nakamoto-3-0 - Handled StacksHTTPRequest - /v2/transactions - 68
10/23/2024, 02:46:03.078 PM	testnet-stacks-blockchain-nakamoto-3-0 - Handled StacksHTTPRequest - /v2/transactions - 105
10/23/2024, 02:46:02.973 PM	testnet-stacks-blockchain-nakamoto-3-0 - Handled StacksHTTPRequest - /v2/transactions - 114
10/23/2024, 02:46:01.856 PM	testnet-stacks-blockchain-nakamoto-3-0 - Handled StacksHTTPRequest - /v3/blocks/c5807470c224c9fe41527bf7f3f90852760cfa9446fdb89e039240b7619864c4 - 0
10/23/2024, 02:46:01.470 PM	testnet-stacks-blockchain-nakamoto-3-0 - Handled StacksHTTPRequest - /v2/transactions - 0
10/23/2024, 02:46:01.168 PM	testnet-stacks-blockchain-nakamoto-3-0 - Handled StacksHTTPRequest - /v2/pox - 116
10/23/2024, 02:46:01.051 PM	testnet-stacks-blockchain-nakamoto-3-0 - Handled StacksHTTPRequest - /v2/accounts/ST2D1M978SCE52GAV07VXSRC9DQBP69X5WHX0DHN5?proof=0 - 64
10/23/2024, 02:46:00.986 PM	testnet-stacks-blockchain-nakamoto-3-0 - Handled StacksHTTPRequest - /v2/accounts/ST484MS3VACPAZ90WHC21XQ7T6XANCV341HJYE0W?proof=0 - 53
10/23/2024, 02:46:00.932 PM	testnet-stacks-blockchain-nakamoto-3-0 - Handled StacksHTTPRequest - /v2/accounts/ST3MNK12DGQF7JN4Q0STK6926VWE5MN21KJ4EGV0E?proof=0 - 52
10/23/2024, 02:46:00.496 PM	testnet-stacks-blockchain-nakamoto-3-0 - Handled StacksHTTPRequest - /v2/transactions - 69
10/23/2024, 02:46:00.426 PM	testnet-stacks-blockchain-nakamoto-3-0 - Handled StacksHTTPRequest - /v2/transactions - 68
10/23/2024, 02:45:59.926 PM	testnet-stacks-blockchain-nakamoto-3-0 - Handled StacksHTTPRequest - /v2/transactions - 109
10/23/2024, 02:45:59.534 PM	testnet-stacks-blockchain-nakamoto-3-0 - Handled StacksHTTPRequest - /v2/transactions - 74
10/23/2024, 02:45:59.458 PM	testnet-stacks-blockchain-nakamoto-3-0 - Handled StacksHTTPRequest - /v2/transactions - 74
10/23/2024, 02:45:59.383 PM	testnet-stacks-blockchain-nakamoto-3-0 - Handled StacksHTTPRequest - /v2/transactions - 74
10/23/2024, 02:45:58.948 PM	testnet-stacks-blockchain-nakamoto-3-0 - Handled StacksHTTPRequest - /v3/blocks/ef7cfd2050ac8b78acb983cd879d0dac12faaf30ab4e3ef400180de87b7809a7 - 0
10/23/2024, 02:45:58.551 PM	testnet-stacks-blockchain-nakamoto-3-0 - Handled StacksHTTPRequest - /v2/transactions - 0
10/23/2024, 02:45:58.551 PM	testnet-stacks-blockchain-nakamoto-3-0 - Handled StacksHTTPRequest - /v2/pox - 118
10/23/2024, 02:45:58.431 PM	testnet-stacks-blockchain-nakamoto-3-0 - Handled StacksHTTPRequest - /v2/info - 0
10/23/2024, 02:45:58.016 PM	testnet-stacks-blockchain-nakamoto-3-0 - Handled StacksHTTPRequest - /v2/accounts/ST484MS3VACPAZ90WHC21XQ7T6XANCV341HJYE0W?proof=0 - 55
10/23/2024, 02:45:57.960 PM	testnet-stacks-blockchain-nakamoto-3-0 - Handled StacksHTTPRequest - /v2/accounts/ST2D1M978SCE52GAV07VXSRC9DQBP69X5WHX0DHN5?proof=0 - 55
10/23/2024, 02:45:57.904 PM	testnet-stacks-blockchain-nakamoto-3-0 - Handled StacksHTTPRequest - /v2/accounts/ST3MNK12DGQF7JN4Q0STK6926VWE5MN21KJ4EGV0E?proof=0 - 50
10/23/2024, 02:45:57.773 PM	testnet-stacks-blockchain-nakamoto-3-0 - Handled StacksHTTPRequest - /v3/tenures/803a02e9d50f7dbdbf71c338dd5cc2a881623e9ba39a57c12b1aa6141136d586 - 19
10/23/2024, 02:45:57.752 PM	testnet-stacks-blockchain-nakamoto-3-0 - Handled StacksHTTPRequest - /v2/transactions - 149
10/23/2024, 02:45:57.602 PM	testnet-stacks-blockchain-nakamoto-3-0 - Handled StacksHTTPRequest - /v2/transactions - 138
10/23/2024, 02:45:57.463 PM	testnet-stacks-blockchain-nakamoto-3-0 - Handled StacksHTTPRequest - /v2/transactions - 130
10/23/2024, 02:45:56.884 PM	testnet-stacks-blockchain-nakamoto-3-0 - Handled StacksHTTPRequest - /v2/transactions - 128
10/23/2024, 02:45:56.754 PM	testnet-stacks-blockchain-nakamoto-3-0 - Handled StacksHTTPRequest - /v2/transactions - 145
10/23/2024, 02:45:56.608 PM	testnet-stacks-blockchain-nakamoto-3-0 - Handled StacksHTTPRequest - /v2/transactions - 157
10/23/2024, 02:45:56.450 PM	testnet-stacks-blockchain-nakamoto-3-0 - Handled StacksHTTPRequest - /v2/transactions - 127
10/23/2024, 02:45:56.321 PM	testnet-stacks-blockchain-nakamoto-3-0 - Handled StacksHTTPRequest - /v2/transactions - 65
10/23/2024, 02:45:56.255 PM	testnet-stacks-blockchain-nakamoto-3-0 - Handled StacksHTTPRequest - /v2/accounts/ST2Q6124HQFKVKPJSS5J6156BJR74FD6EC1297HJ1?proof=0 - 49
10/23/2024, 02:45:56.205 PM	testnet-stacks-blockchain-nakamoto-3-0 - Handled StacksHTTPRequest - /v2/transactions - 103
10/23/2024, 02:45:55.741 PM	testnet-stacks-blockchain-nakamoto-3-0 - Handled StacksHTTPRequest - /v2/transactions - 136
10/23/2024, 02:45:55.605 PM	testnet-stacks-blockchain-nakamoto-3-0 - Handled StacksHTTPRequest - /v2/pox - 119
10/23/2024, 02:45:55.485 PM	testnet-stacks-blockchain-nakamoto-3-0 - Handled StacksHTTPRequest - /v2/accounts/ST1114TBQYGNPGFAVXKWBKZAHP0X7ZGX9K6XYYE4F?proof=0 - 57
10/23/2024, 02:45:55.426 PM	testnet-stacks-blockchain-nakamoto-3-0 - Handled StacksHTTPRequest - /v2/transactions - 127
10/23/2024, 02:45:55.298 PM	testnet-stacks-blockchain-nakamoto-3-0 - Handled StacksHTTPRequest - /v2/accounts/ST20Q2N56E1NBWE37R4VGSF89X4HHTB3GSMD8GKYW?proof=0 - 48
10/23/2024, 02:45:55.249 PM	testnet-stacks-blockchain-nakamoto-3-0 - Handled StacksHTTPRequest - /v2/transactions - 0
10/23/2024, 02:45:55.249 PM	testnet-stacks-blockchain-nakamoto-3-0 - Handled StacksHTTPRequest - /v2/contracts/interface/ST000000000000000000002AMW42H/pox-4 - 49
10/23/2024, 02:45:55.199 PM	testnet-stacks-blockchain-nakamoto-3-0 - Handled StacksHTTPRequest - /v3/tenures/info - 0
10/23/2024, 02:45:55.198 PM	testnet-stacks-blockchain-nakamoto-3-0 - Handled StacksHTTPRequest - /v2/transactions - 156
10/23/2024, 02:45:54.673 PM	testnet-stacks-blockchain-nakamoto-3-0 - Handled StacksHTTPRequest - /v2/transactions - 105
10/23/2024, 02:45:54.463 PM	testnet-stacks-blockchain-nakamoto-3-0 - Handled StacksHTTPRequest - /v3/tenures/0ca0d2f26b6a1d4716884be8f3140cac4752385c5e60314bcb74fca6d38bd788 - 16
10/23/2024, 02:45:54.445 PM	testnet-stacks-blockchain-nakamoto-3-0 - Handled StacksHTTPRequest - /v2/transactions - 65
10/23/2024, 02:45:54.379 PM	testnet-stacks-blockchain-nakamoto-3-0 - Handled StacksHTTPRequest - /v2/contracts/interface/ST000000000000000000002AMW42H/pox-4 - 49
10/23/2024, 02:45:54.328 PM	testnet-stacks-blockchain-nakamoto-3-0 - Handled StacksHTTPRequest - /v2/transactions - 74
10/23/2024, 02:45:54.254 PM	testnet-stacks-blockchain-nakamoto-3-0 - Handled StacksHTTPRequest - /v2/contracts/interface/ST000000000000000000002AMW42H/pox-4 - 49
10/23/2024, 02:45:54.203 PM	testnet-stacks-blockchain-nakamoto-3-0 - Handled StacksHTTPRequest - /v2/accounts/ST2D1M978SCE52GAV07VXSRC9DQBP69X5WHX0DHN5?proof=0 - 53
10/23/2024, 02:45:54.149 PM	testnet-stacks-blockchain-nakamoto-3-0 - Handled StacksHTTPRequest - /v2/accounts/ST484MS3VACPAZ90WHC21XQ7T6XANCV341HJYE0W?proof=0 - 49
10/23/2024, 02:45:54.099 PM	testnet-stacks-blockchain-nakamoto-3-0 - Handled StacksHTTPRequest - /v2/accounts/ST3MNK12DGQF7JN4Q0STK6926VWE5MN21KJ4EGV0E?proof=0 - 49
10/23/2024, 02:45:54.047 PM	testnet-stacks-blockchain-nakamoto-3-0 - Handled StacksHTTPRequest - /v2/transactions - 111
10/23/2024, 02:45:53.934 PM	testnet-stacks-blockchain-nakamoto-3-0 - Handled StacksHTTPRequest - /v2/pox - 122
10/23/2024, 02:45:53.439 PM	testnet-stacks-blockchain-nakamoto-3-0 - Handled StacksHTTPRequest - /v2/transactions - 150
10/23/2024, 02:45:53.288 PM	testnet-stacks-blockchain-nakamoto-3-0 - Handled StacksHTTPRequest - /v2/transactions - 73
10/23/2024, 02:45:53.214 PM	testnet-stacks-blockchain-nakamoto-3-0 - Handled StacksHTTPRequest - /v2/pox - 141
10/23/2024, 02:45:53.205 PM	testnet-stacks-blockchain-nakamoto-3-0 - Cycle #4 Signer #1: received a block proposal for a new block. Submit block for validation.  -  - 
10/23/2024, 02:45:53.141 PM	testnet-stacks-blockchain-nakamoto-3-0 - Cycle #4 Signer #0: received a block proposal for a new block. Submit block for validation.  -  - 
10/23/2024, 02:45:53.072 PM	testnet-stacks-blockchain-nakamoto-3-0 - Handled StacksHTTPRequest - /v2/pox - 144
10/23/2024, 02:45:53.072 PM	testnet-stacks-blockchain-nakamoto-3-0 - Cycle #4 Signer #3: received a block proposal for a new block. Submit block for validation.  -  -

Looks like we have two candidates for network RPC optimization: /v2/transactions and /v2/pox.

@saralab saralab added this to the Nakamoto-3.0.x milestone Oct 30, 2024
@hstove
Copy link
Contributor

hstove commented Nov 14, 2024

I think it would be good to get more clear on what exactly this ticket aims to solve. It's certainly fine for this ticket to be "we have a problem, we should figure out what we want to do about it", but I think we should figure out what that work is and split that into individual tickets.

IMO there are a number of known improvements here:

[signer] Track 429 block submission failures in the database and resubmit them following a block validation response - this is a big one
improve debug logging performance
ignore old block validation requests

It also appears that you've identified two potential big wins:

Looks like we have two candidates for network RPC optimization: /v2/transactions and /v2/pox.

I'd note that the /v2/transactions issue came from Hiro's infra proxying /v2/transactions to all of their nodes, and I believe @CharlieC3 changed that so that the signer node doesn't get those proxied anymore.

@CharlieC3
Copy link
Member

I'd note that the /v2/transactions issue came from Hiro's infra proxying /v2/transactions to all of their nodes, and I believe @CharlieC3 changed that so that the signer node doesn't get those proxied anymore.

Maybe the signals got mixed in the lines somewhere, but it's not quite this. We only mirror /v2/transactions POSTs to 1 additional stacks node -- the one serving events to the Stacks Blockchain API. And the mirror happens asynchronously, so any performance issues on this endpoint are independent of the proxying that happens beforehand.

@hstove
Copy link
Contributor

hstove commented Nov 14, 2024

Ah, I also found the Slack thread where we discussed this, and there was an issue where the stacking script was trying to re-broadcast a stack-extend every ~20 seconds (we've fixed that issue)

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees

@jcnelson jcnelson

Labels
performance-optimizations
Projects
Stacks Core Eng
Status: Status: 🆕 New
Milestone
Nakamoto-3.0.x
Development

No branches or pull requests
4 participants
@saralab @jcnelson @hstove @CharlieC3