Improve quality of socket loop

[huitema]

The initial design of picoquic envisaged that applications would provide their own "socket loop", but we realize now that many would prefer to run picoquic in a background thread. The current "sockloop.c" would work, but it suffers from a few issues:

• there is no way for the app to "wake up" the stack when new data is ready,
• the code is a bit too complex because it includes demonstration logic for demo scenarios, such as simulating multihoming with 2 ports.
• there are two separate loops for Windows and Linux.
• there is not much documentation.
• the time check callback should recompute the time if the application calls the API to insert data

In addition, we need to develop automated testing.

[victorstewart]

if it were Linux only, you could use io_uring and send buffers (which implemented some header + payload message scheme) back and forth between the host and the picoquic loop without any synchronization necessary. this is what i do in my multithreaded processes. all threads run nonstop, harvesting inter-thread communication intertwined in sequence with any other operating system interactions (sendmsg, recvmsg, etc)

[huitema]

Todo -- verify type and use of ifindex -- unsigned int for Linux, ULONG for Windows.

[huitema]

First step is to program a unit test, maybe with a single QUIC context and a loopback socket.

[fbussery]

Just for your information and to share our experience with using picoquic:

During the phase of selecting our QUIC library, we initially overlooked the picoquic library. The main reason was that, for our project, we don't have direct access to the UDP socket. Instead, an external library provides us with a specific API that uses callbacks for UDP communication. Initially, it appeared challenging to integrate this with your socket loop implementation, and we mistakenly believed this loop was mandatory.

By the way, it is not, and I can confirm that there might be a lack of documentation on this aspect. To provide more insight into our implementation, we rely on Boost Asio to manage our own loop.

[huitema]

In some cases we observe diminished performance. The underlying infrastructure could support Gigabit/s but the application struggles sending more than 10 or 20 Mbps. The simplest explanation is that the background thread is starving. This can happen for example if the thread is waiting on a select call for a limited delay, but the select call only returns long after the delay is passed or messages have arrived.

TODO: evaluate whether the socket thread is starving. Maybe by exposing statistics on expected versus actual wait time.

[huitema]

@fbussery could you open a separate issue for the integration with an 'external' event manager? There are several possible options, maybe Boost Asio, maybe libevent, maybe something else. That's a useful topic, but it poses a set of issues different from the "socket loop in a background thread" discussed here.

[fbussery]

@fbussery could you open a separate issue for the integration with an 'external' event manager? There are several possible options, maybe Boost Asio, maybe libevent, maybe something else. That's a useful topic, but it poses a set of issues different from the "socket loop in a background thread" discussed here.

Issue opened: #1608

[huitema]

Fixed.