[dpdk-dev] [PATCH v2] app/test: enhance sanity script

Parthasarathy, JananeeX M jananeex.m.parthasarathy at intel.com
Sun May 20 19:41:50 CEST 2018


Hello Thomas,



-----Original Message-----
From: Thomas Monjalon [mailto:thomas at monjalon.net]
Sent: Friday, May 18, 2018 7:18 PM
To: Parthasarathy, JananeeX M <jananeex.m.parthasarathy at intel.com>
Cc: dev at dpdk.org; Burakov, Anatoly <anatoly.burakov at intel.com>; Pattan, Reshma <reshma.pattan at intel.com>
Subject: Re: [dpdk-dev] [PATCH v2] app/test: enhance sanity script



18/05/2018 15:32, Parthasarathy, JananeeX M:

> From: Thomas Monjalon [mailto:thomas at monjalon.net]

> > 15/05/2018 18:14, Parthasarathy, JananeeX M:

> > > From: Thomas Monjalon [mailto:thomas at monjalon.net]

> >

> > I think we need to keep a fast subset of tests.

> >

> > Please check in mk/rte.sdktest.mk:

> >               PERFLIST=ring_perf,mempool_perf,memcpy_perf,hash_perf,timer_perf

> >         test-fast: BLACKLIST=-$(PERFLIST)

>

> We have added the below test cases to blacklist in mk/rte.sdktest.mk

> reciprocal_division,reciprocal_division_perf,lpm_perf,red_all

>

> Now time taken is around 4mins.

> Please let us know is there any minimum time limit for test execution.

> Is this time duration fine?



4 minutes, it's a bit long for developer oriented tests.

We need to distinguish which tests are enough for quick tests, and which ones can be in another category for full validation purpose.



Please can you make a summary of which tests you are adding, and propose a priority classification.

Thanks


Please find below the test cases newly added to sanity script (autotest_data.py) with priority classification.

P1 - Main test cases
P2 - Cryptodev test cases which requires dependent libraries   (Can be added if the environment is set accordingly)
P3 - Perf test cases and others which takes more time (> 10s)  (these will be in the blacklist test cases)
P4 - Logging/Dump test cases (to be checked whether these are really necessary)

The blacklist is updated in makefile "rte.sdktest.mk" with below list of test cases to reduce the total test execution time.

Blacklist test cases are ring_perf,mempool_perf,memcpy_perf,hash_perf,timer_perf,reciprocal_division,reciprocal_division_perf,lpm_perf,
                  red_all,barrier,hash_multiwriter,timer_racecond,efd,hash_functions,eventdev_selftest_sw,member_perf,efd_perf,lpm6_perf,red_perf,distributor_perf

Currently after the above blacklist update, the total execution time of "make test-fast" is 54s.

Eventdev selftest octeontx

P1

Event ring autotest

P1

Table autotest

P1

Flow classify autotest

P1

Dump struct sizes

P4

Dump mempool

P4

Dump malloc stats

P4

Dump devargs

P4

Dump log types

P4

Dump_ring

P4

Quit

P4

Set rxtx anchor

P4

Set rxtx sc

P4

Event eth rx adapter autotest

P1

Rawdev autotest

P1

Kvargs autotest

P1

Dump physmem

P1

Dump memzone

P4

Devargs autotest

P4

Link bonding autotest

P1

Link bonding mode4 autotest

P1

Link bonding rssconf autotest

P1

Crc autotest

P1

Distributor autotest

P1

Reorder autotest

P1

Barrier autotest

P3

User delay

P1

Bitmap test

P1

Hash scaling autotest

P1

Hash multiwriter autotest

P3

Service autotest

P1

Timer racecond autotest

P3

Member autotest

P3

Efd_autotest

P3

Thash autotest

P1

Hash function autotest

P3

Cryptodev sw mrvl autotest

P2

Cryptodev dpaa2 sec autotest

P2

Cryptodev dpaa sec autotest

P2

Cryptodev qat autotest

P2

Cryptodev aesni mb autotest

P2

Cryptodev openssl autotest

P2

Cryptodev scheduler autotest

P2

Cryptodev aesni gcm autotest

P2

Cryptodev null autotest

P2

Cryptodev sw snow3g autotest

P2

Cryptodev sw kasumi autotest

P2

Cryptodev_sw_zuc_autotest

P2

Reciprocal division

P3

Red all

P3

Mempool performance autotest

P3

Memcpy performance autotest

P3

Hash performance autotest

P3

Timer performance autotest

P3

Reciprocal division perf

P3

Member perf autotest

P3

Efd perf autotest

P3

Lpm perf autotest

P3

Lpm6 perf autotest

P3

Red_perf

P3

Distributor perf autotest

P3

Ring pmd perf autotest

P1

Pmd perf autotest

P1

Ring performance autotest

P1



Thanks

M.P.Jananee


--------------------------------------------------------------
Intel Research and Development Ireland Limited
Registered in Ireland
Registered Office: Collinstown Industrial Park, Leixlip, County Kildare
Registered Number: 308263


This e-mail and any attachments may contain confidential material for the sole
use of the intended recipient(s). Any review or distribution by others is
strictly prohibited. If you are not the intended recipient, please contact the
sender and delete all copies.


More information about the dev mailing list