[dpdk-dev] [Bug 826] red_autotest random failures

[Brandon Lo]

> On Mon, Jan 31, 2022 at 2:27 PM Danilewicz, MarcinX <marcinx.danilewicz at intel.com> wrote:
>> After some time I did some testing. As you may guess, with real hardware I could not reproduce error.
>>
>> From what I see, the problem was here:
>>
>> FT2
>> RED config,     avg queue size, min threshold,  max threshold,  drop prob %,    drop rate %,    diff %,         tolerance %    ,
>> 5              127            32             128            1.6493         0.9900         0.0000         50.0000
>> 6              127            32             128            1.4137         0.8500         0.0000         50.0000
>> 7              127            32             128            1.2370         0.7300         0.0000         50.0000
>> 8              127            32             128            1.0995         0.6200         0.0000         50.0000
>> 9              127            32             128            0.9896         0.6300         0.0000         50.0000
>> ------------------------------------------------------------------------
>>
>> Drop_rate in line 8 should not be greater than in line 9. However by looking at other results, drop_rate value in line 9 is about 0.1% greater than expected. Line 8 results are fine to me.
>>
>> How often any can see this issue? Is there a chance I could use existing docker container for testing?

Hi Marcin,

Attached is an Ubuntu 20.04 Dockerfile that we use in the lab for unit testing.

I don't think we run into the red_autotest failure too often, so it is
hard to debug. My guess is that the test will randomly fail when there
are a lot of processes running on the system, especially since we run
multiple tests per system in the lab.
In a typical worst-case scenario, we can see a machine performing 2 to
3 total compile/ABI tests along with a single unit test. We do
allocate a specific amount of resources (4GB RAM, 2 cores) to each
container, so that can be another factor that affects the frequency of
these failures.

If this issue seems too dependent on the current system load and other
situational factors, it might be good to think about running the
red_autotest separate from other tests in the lab so it does not have
to compete for resources.
Any thoughts on this?

Thanks,
Brandon


-- 
Brandon Lo
UNH InterOperability Laboratory
21 Madbury Rd, Suite 100, Durham, NH 03824
blo at iol.unh.edu
www.iol.unh.edu