An idea that might alleviate the problem: if backoff.is_completed() is true in push/pop retry loops, we should probably register the sender/receiver and park the thread so that the operation we're blocked on wakes us up.

For the purposes of this issue its worth noting that I spoke with @stjepang over discord about the situation.

Your suggestion might be worth exploring. For now I was able to alleviate the problem by attempting to coalesce things coming through the channel:

      loop {
          match sel.ready() {
              i if i == events_ready => {
                  // attempt to wait for more events to enable batch processing and cut down on yields
                  thread::sleep(std::time::Duration::from_nanos(500_000));
                  // use events_ready
              }
              i if i == shutdown_ready => {
                  shutdown.recv()
                  break;
              }
              _ => unreachable!(),
          }
      }

The sleep is not the best solution here but currently is enough to allow the thread to batch things together. It may be better to try something in the backoff module like a short spin lock etc. Really the API that I am trying to mimic here is recv_max_timeout(n: usize, t: Duration). Which ideally receives up to the max specified events or times out before then returning whatever was received in attempt to wait for other things coming down the channel if that makes sense.

I'm thinking the maybe the problem is the call to slot.wait_write() inside read, at

that then uses a backoff without ever blocking if it's completed, at

Also note that this would occur in the context of this

where the "outer" backoff is checked for completeness and the thread blocks, if necessary.

So it appears to that the "inner backoff", the one used inside wait_write, could potentially defeat the completeness check of the outer backoff, the one used in read, with the reading thread stuck in the below loop and calling ::std::thread::yield_now(), while the writing thread is slow to write the message in the block?

I guess the solution could be rewriting the read logic so that it is better plugged into the outer loop and potentially blocks when the backoff completes.

I can suggest the following:

• Have read return an Option<T>, with None indicating the slot is not ready to read from yet.
• Move the token.list.block.is_null() check to outside of read, so that an error can be returned directly by recv.
• Replace Slot.wait_write with a has_been_written(&self) -> bool method, called by read and used to return None if false.

I'll give it a shot actually...

Really the API that I am trying to mimic here is recv_max_timeout(n: usize, t: Duration). Which ideally receives up to the max specified events or times out before then returning whatever was received in attempt to wait for other things coming down the channel if that makes sense.

Why not use something like:

let mut items = vec![];
let timeout = after(duration);
loop {
    select! {
        recv(timeout) -> _ => { break; }
        recv(chan) -> msg {
            items.push(msg);
            if items.len() >= BATCH_LEN { break; }
        }
    }
}
// Process items...

@papertigers could you please try running your test with this PR and see if it helped at all? #447