[dpdk-dev] [EXT] [PATCH v4 0/7] test: refactor crypto unit test framework
Thomas Monjalon
thomas at monjalon.net
Wed May 12 17:42:13 CEST 2021
I see this patchset is merged in the crypto tree but I guess it is not definitive.
Am I right it is too late to do such refactoring after -rc2?
What is the intent?
12/05/2021 15:15, Akhil Goyal:
> Hi Ciara,
>
> I see that CI is reporting one error in autotest. Can you please check if it is because of your patchset?
>
> 9/96 DPDK:fast-tests / cycles_autotest FAIL 1.42s (exit status 255 or signal 127 SIGinvalid)
> 12:33:15 DPDK_TEST=cycles_autotest MALLOC_PERTURB_=44 /home/jenkins-local/jenkins-agent/workspace/Ubuntu18.04-Unit-Test-DPDK/dpdk/build/app/test/dpdk-test '-l 0-3' --file-prefix=cycles_autotest
> ----------------------------------- output -----------------------------------
> stdout:
> RTE>>cycles_autotest
>
> delay_us is not accurate: too long
> Test Failed
>
>
> Regards,
> Akhil
>
> > -----Original Message-----
> > From: Ciara Power <ciara.power at intel.com>
> > Sent: Wednesday, May 12, 2021 5:07 PM
> > To: dev at dpdk.org
> > Cc: declan.doherty at intel.com; Akhil Goyal <gakhil at marvell.com>;
> > aconole at redhat.com; hemant.agrawal at nxp.com; Anoob Joseph
> > <anoobj at marvell.com>; ruifeng.wang at arm.com; asomalap at amd.com;
> > ajit.khaparde at broadcom.com; g.singh at nxp.com; roy.fan.zhang at intel.com;
> > Ciara Power <ciara.power at intel.com>
> > Subject: [EXT] [PATCH v4 0/7] test: refactor crypto unit test framework
> >
> > External Email
> >
> > ----------------------------------------------------------------------
> > The current crypto unit test framework is not granular enough to
> > accurately track unit test results. This is caused by one testcase
> > in a suite actually running multiple testcases, but only returning
> > one result.
> >
> > The approach taken in this patchset allows a test suite have a
> > list of sub-testsuites, and/or a list of testcases as previously used.
> > The unit test suite runner can then recursively iterate and run the
> > sub-testsuites, until it reaches a suite with testcases,
> > and it then runs each testcase as it had done previously.
> > In the case of a testsuite with both testcases and sub-testsuites,
> > the testcases are executed first before iterating through the
> > sub-testsuites.
> >
> > By allowing this further breakdown into sub-testsuites,
> > a refactor of the crypto unit tests solves the issue of inaccurate
> > reporting, as sub-testsuites can be used in place of the testcases
> > that had multiple testcases hidden on a sub level.
> > The blockcipher tests previously had these hidden testcases,
> > but are now sub-testsuites that are dynamically created and added to a
> > parent test suite, allowing for each testcase status to be reported
> > directly to the runner.
> > The cryptodev test suite is broken down into smaller suites that are
> > used as sub-testsuites, which allows for more flexibility choosing which
> > sub-testsuites should run for the current device.
> > The introduction of sub-testsuites also allows for more precise
> > setup/teardown functions, that can check the capabilities required to
> > run its testcases.
> >
> > For example, when running the cryptodev_aesni_mb_autotest,
> > the parent Cryptodev Test Suite is executed.
> > Various sub-testsuites are added to the parent test suite, such as
> > the static suites of testcases that were once in the cryptodev_testsuite,
> > and blockcipher suites.
> > The unit test runner can then run the Cryptodev parent test suite,
> > which in turn will run the sub-testsuites.
> >
> > The user is now required to create vdevs via EAL commandline args,
> > this is no longer done within the test app for crypto autotests.
> >
> > Documentation will need to be added at a later stage,
> > adding to the test document that isn't yet merged. [1]
> >
> > [1] https://urldefense.proofpoint.com/v2/url?u=https-
> > 3A__patchwork.dpdk.org_project_dpdk_patch_20210309155757.615536-
> > 2D1-2Daconole-
> > 40redhat.com_&d=DwIDAg&c=nKjWec2b6R0mOyPaz7xtfQ&r=DnL7Si2wl_PR
> > wpZ9TWey3eu68gBzn7DkPwuqhd6WNyo&m=XItKx7UnOATOiVC3kRLlV3Woh
> > rLeZbmlqShu_py452U&s=BpOMigsNnvWMV739-
> > fMxEvVERJieNSnfnnmZdiMLMJg&e=
> >
> >
> > v4:
> > - Fixed some return value bugs related to patch 4.
> > - Rebased onto main.
> > v3:
> > - Added support for a testsuite having both a list of testcases,
> > and a list of sub-testsuites.
> > - Replaced PMD based parent testsuites with a cryptodev testsuite
> > used by all autotests, with the exception of scheduler autotest.
> > - Setup functions were added for all sub-testsuites, within which
> > required capability support is checked.
> > - The setup functions no longer create vdevs if needed,
> > this must be done by the user when running the test.
> > - Patch added to standardise return values for skipped testcases.
> > v2:
> > - Added macro in place of testcase/testsuite loops.
> > - Added more detail in the summary output.
> > - Moved testcase counts to the testsuite structure.
> > - Flattened testsuite structure to remove union.
> > - Added patch for fix of blockcipher test return value.
> > - Squashed release note into last patch.
> >
> > Ciara Power (7):
> > app/test: refactor of unit test suite runner
> > test: introduce parent testsuite format
> > test/crypto: refactor to use sub-testsuites
> > test/crypto: replace unsupported with skipped
> > test/crypto: move testsuite params to header file
> > test/crypto: fix return value on test skipped
> > test/crypto: dynamically build blockcipher suite
> >
> > app/test/test.c | 176 +-
> > app/test/test.h | 23 +-
> > app/test/test_cryptodev.c | 2333 ++++++++++++++----------
> > app/test/test_cryptodev.h | 32 +
> > app/test/test_cryptodev_asym.c | 111 +-
> > app/test/test_cryptodev_blockcipher.c | 423 ++++-
> > app/test/test_cryptodev_blockcipher.h | 12 +-
> > app/test/test_ipsec.c | 32 +-
> > doc/guides/rel_notes/release_21_05.rst | 5 +
> > 9 files changed, 2004 insertions(+), 1143 deletions(-)
> >
> > --
> > 2.25.1
>
>
More information about the dev
mailing list