Multi-threaded iperf3 public beta 1

[bmah888]

We're pleased to announce the PUBLIC BETA availability of a multi-threaded version of iperf3.

iperf3 was originally designed as a single-threaded program. Unfortunately, as network speeds increased faster than CPU clock rates, this design choice meant that iperf3 became incapable of using the bandwidth of the links in its intended operating environment (high-performance R&E networks with Nx10Gbps or Nx100Gbps network links and paths).

We have created a variant of iperf3 that uses a separate thread (pthread) for each test stream. As the streams run more-or-less independently, this should remove some of the performance bottlenecks, and allow iperf3 to perform higher-speed tests, particularly on 100+Gbps paths. This version has recorded transfers as high as 148Gbps in internal testing at ESnet.

To try this out, download the following release tarball (with SHA256 hash as shown):

https://downloads.es.net/pub/iperf/iperf-3.13-mt1.tar.gz
581f4485afb941b18f50833940cc021c40f301be104a2e514fc95b2be5faecdf iperf-3.13-mt1.tar.gz

Alternatively, clone the iperf3 repository as usual from GitHub (https://github.com/esnet/iperf) and switch to the mt branch.

The -P/--parallel option will vary the number of parallel streams as usual. However, multi-threaded iperf will allocate a separate thread to every stream/connection (on both client and server).

Most of the testing to date has taken place on Linux x86_64 bare-metal systems, although we've tested for basic operation on FreeBSD and macOS as well. Single-threaded and multi-threaded iperf3 should inter-operate seamlessly.

We intend, if all goes well, to merge this branch to the iperf3 mainline at some point in the future (still to be determined).

We're interested in hearing about people's experiences. Probably the best way is to find the appropriate topic in the GitHub discussions area and add posts to it.

We're especially interested in these areas for testing:

All platforms: Best TCP and UDP performance on high-speed paths (100+Gbps).

Virtual machines, particularly those with small numbers of virtual CPUs: We expect to see poor performance if the number of parallel streams exceeds the number of virtual CPUs, due to threads thrashing between CPU cores.

32-bit CPUs (for example i386 and 32-bit arm): Multi-threaded iperf3 now uses atomic variables to hold various statistics and counters. On our test hosts, these are able to run without locking, so the various threads run more-or-less independently. We're not sure how well this
translates to 32-bit hosts, which might need locking for these atomic variables.

Non-Linux/UNIX hosts: Although we stand by our statement of support for Linux, macOS, and FreeBSD hosts only, we don't want to gratuitously break iperf3 on other platforms.

Possible regression: We've observed some small (3-5%) slowdowns with single-stream TCP performance compared to the stock iperf3.

Documentation: The iperf3 documentation in the beta still corresponds to iperf-3.13, and doesn't reflect any of the changes made for multi-threading.