errors in communication between PC and ARM64 board

[zhangshuai75]

When the ARM64 board is used as the server and the PC is used as the client, the packet loss rate is 0;
But when the PC is used as the server and ARM64 is used as the client, the packet loss rate reaches 53%.

The version of iperf 3.11 is used on the ARM64 board, and the latest version under Windows is iperf 3.1.3
The test commands used are as follows:
Server side ./iperf3 -s -p 1234 -4
Client side ./iperf3 -c 10.169.86.100 -p 1234 -4 -u -b 1000M -t 600

I wonder if it is because of the different default configurations between different versions and different platforms.
By default, the size of the data packet sent when the PC is used as a client is 8192 bytes, and the size of the data packet sent when the board is used as a client is 1460 bytes.
The high packet loss rate occurs when the PC as the server and receives 1460byte data packets.

The test data is as follows:
When PC is used as server and ARM64 is used as client

C:\Users\Downloads\iperf-3.1.3-win64>iperf3.exe -s -p 1234 -4 -V
iperf 3.1.3
CYGWIN_NT-10.0 CN-L-7R2GKR3 2.5.1(0.297/5/3) 2016-04-21 22:14 x86_64
Server listening on 1234
Time: Tue, 05 Dec 2023 09:53:18 GMT
Accepted connection from 10.169.86.100, port 60328
      Cookie: k3dtwj2mdhpa5m4l4mmgdqtlhzmud7wffrti
[  5] local 10.169.86.103 port 1234 connected to 10.169.86.100 port 56062
Starting Test: protocol: UDP, 1 streams, 1460 byte blocks, omitting 0 seconds, 600 second test
[ ID] Interval           Transfer     Bandwidth       Jitter    Lost/Total Datagrams
[  5]   0.00-1.00   sec  71.9 MBytes   603 Mbits/sec  0.021 ms  23984/75613 (32%)
[  5]   1.00-2.00   sec  62.6 MBytes   525 Mbits/sec  0.005 ms  36331/81268 (45%)
[  5]   2.00-3.00   sec  81.4 MBytes   683 Mbits/sec  0.006 ms  23303/81779 (28%)
[  5]   3.00-4.00   sec  75.0 MBytes   629 Mbits/sec  0.016 ms  27973/81820 (34%)
[  5]   4.00-5.00   sec  79.7 MBytes   669 Mbits/sec  0.017 ms  24647/81917 (30%)
[  5]   5.00-6.00   sec  70.8 MBytes   594 Mbits/sec  0.019 ms  30585/81410 (38%)
[  5]   6.00-7.00   sec  57.2 MBytes   480 Mbits/sec  0.031 ms  40561/81641 (50%)
[  5]   7.00-8.00   sec  57.3 MBytes   481 Mbits/sec  0.038 ms  40192/81373 (49%)
[  5]   8.00-9.00   sec  49.3 MBytes   414 Mbits/sec  0.042 ms  46621/82030 (57%)
[  5]   9.00-10.00  sec  60.1 MBytes   504 Mbits/sec  0.022 ms  38725/81914 (47%)
[  5]  10.00-11.00  sec  66.2 MBytes   556 Mbits/sec  0.024 ms  34120/81689 (42%)
[  5]  11.00-12.00  sec  55.6 MBytes   467 Mbits/sec  0.009 ms  41801/81743 (51%)
......
[  5] 594.00-595.00 sec  52.4 MBytes   439 Mbits/sec  0.015 ms  44475/82076 (54%)
[  5] 595.00-596.00 sec  56.2 MBytes   472 Mbits/sec  0.020 ms  41286/81683 (51%)
[  5] 596.00-597.00 sec  52.6 MBytes   441 Mbits/sec  0.070 ms  43815/81608 (54%)
[  5] 597.00-598.00 sec  40.6 MBytes   341 Mbits/sec  0.044 ms  52726/81888 (64%)
[  5] 598.00-599.00 sec  46.0 MBytes   386 Mbits/sec  0.047 ms  48128/81190 (59%)
[  5] 599.00-600.00 sec  37.1 MBytes   311 Mbits/sec  0.023 ms  55651/82261 (68%)
[  5] 600.00-600.04 sec  1.66 MBytes   401 Mbits/sec  0.022 ms  1669/2859 (58%)

Test Complete. Summary Results:
[ ID] Interval           Transfer     Bandwidth       Jitter    Lost/Total Datagrams
[  5]   0.00-600.04 sec  0.00 Bytes  0.00 bits/sec  0.022 ms  27362966/48969460 (56%)
CPU Utilization: local/receiver 77.2% (25.8%u/51.4%s), remote/sender 0.0% (0.0%u/0.0%s)
iperf 3.1.3

When the ARM64 board is used as the server and the PC is used as the client

root@root:# ./iperf3 -s -p 1234 -4 -V
iperf 3.11
Linux 6.1.33-00002-g1653592bb2c7-dirty #16 SMP PREEMPT Fri Dec  1 18:01:43 CST 2023 aarch64
Server listening on 1234 (test #1)
Time: Tue, 05 Dec 2023 17:14:34 GMT
Accepted connection from 10.169.86.103, port 49728
      Cookie: CN-L-7R2GKR3.1701767775.564896.53d35
      Target Bitrate: 1000000000
[  5] local 10.169.86.101 port 1234 connected to 10.169.86.103 port 63920
Starting Test: protocol: UDP, 1 streams, 8192 byte blocks, omitting 0 seconds, 600 second test, tos 0
[ ID] Interval           Transfer     Bitrate         Jitter    Lost/Total Datagrams
[  5]   0.00-1.00   sec  94.3 MBytes   791 Mbits/sec  0.050 ms  0/12073 (0%)
[  5]   1.00-2.00   sec   108 MBytes   903 Mbits/sec  0.021 ms  0/13782 (0%)
[  5]   2.00-3.00   sec   104 MBytes   869 Mbits/sec  0.044 ms  0/13262 (0%)
[  5]   3.00-4.00   sec   107 MBytes   895 Mbits/sec  0.022 ms  0/13662 (0%)
[  5]   4.00-5.00   sec   105 MBytes   885 Mbits/sec  0.071 ms  0/13500 (0%)
[  5]   5.00-6.00   sec   106 MBytes   888 Mbits/sec  0.032 ms  0/13549 (0%)
[  5]   6.00-7.00   sec   107 MBytes   900 Mbits/sec  0.027 ms  0/13736 (0%)
[  5]   7.00-8.00   sec   109 MBytes   916 Mbits/sec  0.019 ms  0/13975 (0%)
......
[  5] 589.00-590.00 sec  99.4 MBytes   834 Mbits/sec  0.022 ms  0/12720 (0%)
[  5] 590.00-591.00 sec   105 MBytes   879 Mbits/sec  0.025 ms  0/13406 (0%)
[  5] 591.00-592.00 sec   108 MBytes   904 Mbits/sec  0.023 ms  0/13801 (0%)
[  5] 592.00-593.00 sec   106 MBytes   892 Mbits/sec  0.182 ms  0/13617 (0%)
[  5] 593.00-594.00 sec   108 MBytes   907 Mbits/sec  0.020 ms  0/13842 (0%)
[  5] 594.00-595.00 sec   108 MBytes   904 Mbits/sec  0.018 ms  0/13795 (0%)
[  5] 595.00-596.00 sec   108 MBytes   903 Mbits/sec  0.018 ms  0/13774 (0%)
[  5] 596.00-597.00 sec   106 MBytes   887 Mbits/sec  0.020 ms  0/13540 (0%)
[  5] 597.00-598.00 sec   109 MBytes   917 Mbits/sec  0.026 ms  0/13993 (0%)
[  5] 598.00-599.00 sec   106 MBytes   885 Mbits/sec  0.027 ms  0/13510 (0%)
[  5] 599.00-599.97 sec   104 MBytes   895 Mbits/sec  0.156 ms  0/13309 (0%)

Test Complete. Summary Results:
[ ID] Interval           Transfer     Bitrate         Jitter    Lost/Total Datagrams
[  5] (sender statistics not available)
[  5]   0.00-599.97 sec  60.5 GBytes   866 Mbits/sec  0.156 ms  0/7929760 (0%)  receiver
iperf 3.11

[davidBar-On]

There may be several reasons for the issue. Network throughput, CPU performance, IP buffers size, etc. Therefore, a main question is whether the issue is because of resources limitation in the client's size. To determine whether the client sent the lost packets or as not able to send them (e.g. because of buffers size) tools like Wireshark, tcpdump can be used to evaluate what packets are going over the network.

Using TCP without throughput limitation can help understand the problem, as such TCP test throughput should be close to the maximum possible rate. Packet size should be the same as in UDP tests, i.e. set -l 1460 (the default TCP packet size is 128KB). Measuring the CPU usage during the TCP test (at least on the ARM64side) can indicate whether there may be CPU performance problem.

By the way, note that 3.1.3 is very old version and it is highly recommended to use a newer version, e.g. from here.

[bmah888]

You're using UDP here. The default UDP send size changed sometime around iperf-3.2 because the old value of 8192 was larger than the maximum packet size on many Ethernet subnets...this caused fragmentation and often increased packet loss. The default value is derived from the maximum packet size as seen from the operating system, and general works out to (as you saw) 1460 bytes.

I'm going to echo @davidBar-On's advice to use a newer version of iperf3. There have been a number of changes in 3.2 and later that improve the experience when doing UDP tests. The current version of iperf3 is 3.16.

[zhangshuai75]

Thanks both for your replies.
I tested it using the new Windows version of the program, and there is still a problem.
However, I found that there will be a lot of packet loss when using two PCs to communicate;
there is no packet loss when using two ARM64 boards to communicate.
So I suspect it is a PC configuration problem, not a problem with the ARM64 board.