Skip to content

Commit

Permalink
Improve runloop observability (#2940)
Browse files Browse the repository at this point in the history
# Description
I tried playing around with the `sync-to-blockchain` mode to see if it
works and noticed a few observability issues

# Changes
* adjusted `current_block_delay` buckets to be more useful (ideally all
measurements should fit into the 0.5s bucket)
* omitted block delay warning in case we finish the prev run loop within
the same block due to missing solutions
* returned correct auction block when awaiting next block so that the
delay metric should be within a few 100ms instead of 12.5s
  • Loading branch information
MartinquaXD authored Sep 4, 2024
1 parent 41334ea commit bef7d2a
Showing 1 changed file with 9 additions and 6 deletions.
15 changes: 9 additions & 6 deletions crates/autopilot/src/run_loop.rs
Original file line number Diff line number Diff line change
Expand Up @@ -101,10 +101,13 @@ impl RunLoop {
let current_block = *self.eth.current_block().borrow();
let time_since_last_block = current_block.observed_at.elapsed();
let auction_block = if time_since_last_block > self.max_run_loop_delay {
tracing::warn!(
missed_by = ?time_since_last_block - self.max_run_loop_delay,
"missed optimal auction start, wait for new block"
);
if prev_block.is_some_and(|prev_block| prev_block != current_block.hash) {
// don't emit warning if we finished prev run loop within the same block
tracing::warn!(
missed_by = ?time_since_last_block - self.max_run_loop_delay,
"missed optimal auction start, wait for new block"
);
}
ethrpc::block_stream::next_block(self.eth.current_block()).await
} else {
current_block
Expand All @@ -118,7 +121,7 @@ impl RunLoop {
{
tracing::warn!(?err, "failed to update auction");
}
current_block
auction_block
}
};

Expand Down Expand Up @@ -862,7 +865,7 @@ struct Metrics {

/// Time difference between the current block and when the single run
/// function is started.
#[metric(buckets(0, 2, 4, 6, 8, 10, 12, 14, 16, 18, 20, 22, 25, 30, 40))]
#[metric(buckets(0.5, 1, 1.5, 2, 4, 6, 8, 10, 12, 16))]
current_block_delay: prometheus::Histogram,
}

Expand Down

0 comments on commit bef7d2a

Please sign in to comment.