Periodic throughput drops in uplink traffic with iperf3 between OFDMA-capable terminals and APs

[BowinChow]

iperf version: 3.13

client type: xiaomi 13 cellphone (Android)

AP brand: NA

Details of question:
We are using15 android clients to make UL traffic to the AP, and the terminal and AP negotiated OFDMA capability. The measurement tool is iperf, in this case, we found that the uplink traffic has a periodic drop, and the parameters used in iperf are all default parameters. If the buffer size (-l) in iperf is changed to 32k, the drop phenomenon is alleviated.

In addition, using IxChariot for upstream streaming also does not cause traffic drop（the buffer size is 32k, default parameter）. Could you share me what is the difference between iperf streaming when the length is 32k and 128k? (I have compared different buffer size parameters of "-l" in the following figure)

[davidBar-On]

Hi @BowinChow, can you please add some more details:

1. I assume by "drop" you mean "throughput drop" and not "packets drop". Is this correct?
2. Did you use UDP or TCP? With UDP the default iperf3 message length (the -l parameter) is 128KB and for UDP about 1,460 bytes.
3. What is the first diagram vs. the second diagram? Is the first shows the throughput of a single device while the second is the average of several devices? If this is the case, how many devices were used for the second diagram?

Regarding your questions, in general, with -l 128K iperf3 is loading the CPU less than with -l 32K, so the throughput is usually higher.

[BowinChow]

Hi David,
Thank you for the kind reply. I will confirm your questions and make supplement of the background.

I assume by "drop" you mean "throughput drop" and not "packets drop". Is this correct?

--Yes, it is throughput drop here.

Did you use UDP or TCP? With UDP the default iperf3 message length (the -l parameter) is 128KB and for UDP about 1,460 bytes.

--We used the TCP communication model for UL-traffic testing in an interference-free environment

What is the first diagram vs. the second diagram? Is the first shows the throughput of a single device while the second is the average of several devices? If this is the case, how many devices were used for the second diagram?

--The first diagram corresponds to mean UL-throughput of 15 Android phones(brand: xiaomi 13) in 300s with default value of -l (128K). The second diagram corresponds to mean UL-throughput of 15 android devices in 300s under different -l options (16, 32, 64, 128K). Both diagram used 15 clients. As a result, throughput oscillations and drops are more pronounced in the scenarios of 128K. The vertical axis of all schematics represents throughput, the horizontal axis represents time, and the curves are the mean throughput of 15 clients in this case.

Regarding your questions, in general, with -l 128K iperf3 is loading the CPU less than with -l 32K, so the throughput is usually higher.

--According to diagram 2, 128K does have a somewhat higher throughput than 32K, but it's not particularly noticeable in this case. Changing the -w parameter varies the throughput very significantly.

More background on this problem:
We used ixchariot and iperf to transmit UL-traffic to APs in an interference-free environment using 15 terminals. Both terminals and APs simultaneously support the OFDMA feature.
What we found:

1. When OFDMA is turned on, iperf's default configuration results in drops and oscillations; Changing the -l parameter to 32K reduces or even eliminates the drops and oscillations; However, ixchariot does not experience these problems with the default parameters.
2. We have made test with different values of -l parameters shown in the second diagram, found that throughput looks like to be odd in -l 128K (default value) compared to other values.

So we have compared ixchariot and iperf in terms of their TCP communication models: ixchariot's tcp default send size is 31K, but we are not sure if this parameter corresponds to the -l parameter in iperf 3.13.

Besides, I have another question about the parameters of iperf3.13.
I am very confused about the difference between "-l" and "-W" options in iperf 3.13. I have read the notes related in iperf3 manuals as:
-l, --length n[KM] The length of buffers to read or write. iPerf works by writing an array of len bytes a number of times. Default is 128 KB for TCP, 8 KB for UDP. See also the -n, k and -t options.

-w, --window n[KM] Sets the socket buffer sizes to the specified value. For TCP, this sets the TCP window size. (this gets sent to the server and used on that side too)

Does "-l" mean the length of data to be sent? And "-w" means the buffer allocated by kernel for per socket? (corresponding to the SO_RCVBUF and SO_SNDBUF parameters).

What would happen if the value of parameter "-l" exceeds the one of "-w"?

Appreciate it if you could kindly reply to me.

BR.
Bowin

[davidBar-On]

Hi @BowinChow,

The first diagram corresponds to mean UL-throughput of 15 Android phones(brand: xiaomi 13) in 300s with default value of -l (128K). The second diagram corresponds to mean UL-throughput of 15 android devices in 300s under different -l options (16, 32, 64, 128K). Both diagram used 15 clients.

I still don't understand what is the first chart. According to your answer, my understanding is that chart 1 and the l_128 graph in chart 2 should have been the same. What I am missing?

We used ixchariot and iperf to transmit UL-traffic to APs in an interference-free environment using 15 terminals. Both terminals and APs simultaneously support the OFDMA feature.

Not sure if this will help, but it is not clear were is the server running. I understand that the iperf3/ixchariot clients are running on the Android device. Are the servers also running on Android devices? On the AP? On a server (the endpoint-2 Windows machine?)?

Does "-l" mean the length of data to be sent? And "-w" means the buffer allocated by kernel for per socket? (corresponding to the SO_RCVBUF and SO_SNDBUF parameters).

Yes, in general. -w sets both SO_RCVBUF and SO_SNDBUF socket parameters. -l is the size of the buffer that iperf3 sends to the socket. Each time the iperf3 code sends data, it sends the -l size. That data is loaded into the socket buffers and the TCP stack is send the data from there.

By the way, --zerocopy reduces the overhead of the data copy between the buffers, and you can try to see if it makes any change.

What would happen if the value of parameter "-l" exceeds the one of "-w"?

I belive that the socket buffer is larger than the -w, and that this value mainly effects the window size of TCP. I.e. the size of size of the allowed unacknoleged packets. Analysing the traffic using tools like Wireshark can help to determine what actually happens.

It may be that -l 128K has more drops because the TCP stack can tries to send too much data at once, causing TCP or network congestion. Again, traffic analysis using Wireshark should be used to determine whether this is the case.

Regarding congestion, one of the differences between ixchariot and iperf3 may be the congestion algorithm used. When using -V option in iperf3, at the end of the test it prints the congestion algorithm used (e.g. "snd_tcp_congestion cubic"). If the algorithms used by the two tool is different, you can try setting the iperf3 congestion algorithm to the one used by ixchariot, using the --congestion option.

[BowinChow]

Hi David,

I still don't understand what is the first chart. According to your answer, my understanding is that chart 1 and the l_128 graph in chart 2 should have been the same. What I am missing?

Sorry for my unclear descriptions. The yellow curve in the first figure and the blue curve in the second figure are test observations at different times in the same environment, so they don't have the same throughput, but both show oscillations and drops in general.

Not sure if this will help, but it is not clear were is the server running. I understand that the iperf3/ixchariot clients are running on the Android device. Are the servers also running on Android devices? On the AP? On a server (the endpoint-2 Windows machine?)?

We have run the ixchariot and iperf on a PC server and the clients are android phones.

Thank you for sharing us these details. I will make more tests according to your advice. I will share the results in this discussion if any progress available.

Besides, I have a suspicion that this is also somewhat related to the working mechanism of OFDMA: OFDMA divides the channel bandwidth into RUs to split packets into combinations, for -l 128k packets compared to -l 32K packets are likely to be larger, which may bring about additional scheduling overhead in OFDMA, which in itself mainly reduces the latency of the data, and may not be able to increase the throughput of the larger packets.

Thanks.

BR.
Bowin