PHY Conformance Tests

[dvdgrgrtt]

Hi there!

Have you ever wondered how we assess the performance of the srsRAN gNB and ensure that it complies with the requirements of the 3GPP and O-RAN specifications? Well, as you can imagine, there is a lot of testing involved that includes both simulated/emulated set-ups (e.g., with the VIAVI Network Tester) and over-the-air RF transmission with COTS terminals. The extent of the test may also vary from a specific component that is evaluated in isolation, to system-wise experiments that deal with the whole gNB.

In this post we are having a look at a set of simple tests for the uplink channels (PUSCH, PUCCH and PRACH). The purpose of these tests is to assess the ability of the srsRAN gNB to receive uplink transmissions and meet the requirements specified by the 3GPP in TS38.104 and TS38.141. As we'll se next, these tests rely on the MATLAB 5G Toolbox to simulate the uplink transmission and then use the srsRAN components, included via MEX functions, to receive it. Of course, this approach introduces some simplifications over a full hardware set-up (notably, everything happens in the baseband domain, without an RU model) and the results are probably somehow optimistic. However, it gives us a pretty good idea of the behavior of the uplink processors. Moreover, it is a very flexible testing environment, easy to deploy and extend thanks to all the 3GPP-compliant MATLAB functions, and easily replicable by the srsRAN Project community.

How it works

The general block diagram of the conformance tests is depicted in the figure below. Blue blocks are implemented in MATLAB, while the orange block represents the srsRAN UL component under test, which is included thanks to a simple MEX wrapper.

In few words, after configuring all the blocks, we enter a loop in which we generate a random UL/RACH transmission, we simulate the channel impairments (3GPP TDL fading model and AWGN) and we feed the received signal to the component under test. The loop continues until we have enough data for an accurate estimation of the performance metric we are interested in (e.g., throughput or false alarm probability).

At the end of the test, the performance metric evaluated by simulation is compared with the target value provided by the TSs for each specific configuration, giving us an idea of how well the UL processors are working.

Try it yourself!

If you have access to MATLAB, trying the conformance tests yourself is straightforward.

First, clone our companion srsRAN-matlab repository and follow the instructions to compile the MEX libraries of the srsRAN components. Next, launch MATLAB and move to the srsran_matlab folder. From there, you can run the following commands:

% Add the srsRAN-matlab folder to the MATLAB search path for easy access
% to all the functionalities.
addpath .

% Run the PUSCH conformance tests.
runtests('unitTests/CheckPUSCHConformance.m')

% Run the PUCCH Format 0 conformance tests.
runtests('unitTests/CheckPUCCHF0Conformance.m')

% Run the PUCCH Format 1 conformance tests.
runtests('unitTests/CheckPUCCHF1Conformance.m')

% Run the PUCCH Format 2 conformance tests.
runtests('unitTests/CheckPUCCHF2Conformance.m')

% Run the PRACH conformance tests.
runtests('unitTests/CheckPRACHConformance.m')

% Run all conformance tests with a single command.
runtests('unitTests', tag='conformance')

Warning: some of the tests (e.g., CheckPRACHConformance/checkPRACHfalsealarm) are very long and may take several hours to complete.

Results

As explained above, the tests are designed mainly to check whether we pass or fail when comparing the evaluated performance metric with the target value in the TSs, and provide little information. The output of a successful test will look something like

Running CheckPUSCHConformance
Simulating transmission scheme 1 (1x2) and SCS=30kHz with TDLB100 channel at -2.3dB SNR for 200 10ms frame(s)
SRS
Throughput(Mbps) after 200 frame(s) = 2.2044 (max 2.6400)
Throughput(%) after 200 frame(s) = 83.5000
BLER after 200 frame(s) = 0.1955
.

If a test fails, on the other hand, the output will look like

Running CheckPRACHConformance
10:06:13: Simulating SNR = -14.5 dB...
False-alarm probability: 0.10%
.
10:36:16: Simulating SNR = -6.6 dB...
False-alarm probability: 0.11%
================================================================================
Verification failed in CheckPRACHConformance/checkPRACHfalsealarm(TestConfig=struct_1).
----------------
Test Diagnostic:
----------------
WARNING: The PRACH false detection rate should not be higher than 0.1%.
---------------------
Framework Diagnostic:
---------------------
verifyLessThanOrEqual failed.
--> The value must be less than or equal to the maximum value.
Actual Value:
0.001140000000000
Maximum Value (Inclusive):
1.000000000000000e-03
------------------
Stack Information:
------------------
In /builds/softwareradiosystems/srsgnb_matlab/unitTests/CheckPRACHConformance.m (CheckPRACHConformance.checkPRACHfalsealarm) at 101
================================================================================
.
11:15:41: Simulating SNR = -16.7 dB...
False-alarm probability: 0.09%
.11:47:37: Simulating SNR = -11.9 dB...
False-alarm probability: 0.10%
.
Done CheckPRACHConformance
__________
Failure Summary:
    Name                                                          Failed Incomplete Reason(s)
    ==============================================================================================================
    CheckPRACHConformance/checkPRACHfalsealarm(TestConfig=struct_1) X               Failed by verification.

If you'd like to get some more information about the behavior of the UL components and get their performance under different conditions than the ones specified by the TSs for the conformance tests (e.g., draw a BLER curve as a function of the SNR), you can use our simulators (see the "Simulators" tab our srsRAN-matlab tutorial). Indeed, the tests above are just an automated call to the corresponding simulator with the right configuration. Just remember to set ImplementationType = 'srs' to be sure to run the srsRAN version of the UL component (as opposed to the MATLAB one from the 5G Toolbox) via the MEX library.