[EXTERNAL] Re: [RFC PATCH 3/3] dts: add cryptodev testsuite

[Andrew Bailey]

 have a few more questions for you regarding testing cryptodev. As a
reminder, we're developing a test suite to detect performance or functional
regressions in cryptodev, using the dpdk-test-crypto-perf application with
Throughput, Latency, and Verify ptest options. Right now we are testing
with an Intel QAT 8970. It has 3 physical functions, and DTS automatically
creates 16 VFs on each PF when the testrun starts. I have some questions
regarding which VFs to use and how many to use when checking for
regressions:


   - Is it appropriate to specify only a single virtual function in the
   allowed ports list for our test suite?
   - Under what circumstances should we allow multiple virtual functions?
   - When allowing multiple virtual functions, should we include one from
   each physical function?


Thank you for your guidance.

On Mon, Jan 12, 2026 at 2:20 PM Akhil Goyal <gakhil at marvell.com> wrote:

> Hi Andrew,
>
>    1. Which metrics in the attached table below are relevant for checking
>    for performance regressions in throughput testing? MOps? Gbps?
>
> Mops and Gbps both are calculation based(Mops * packet_size = Gbps). You
> can use anyone of them.
>
>    2. What would be an acceptable throughput deviation between runs, and
>    what percentage deviation should be considered a failure? For instance,
>    right now I am thinking that if the test returns a Gbps reduction of 2% or
>    greater from the given baseline, that will trigger a failure.
>
> IMO, 5% should be good enough.
>
>
>
> Regards,
>
> Akhil
>
>
>
> *From:* Andrew Bailey <abailey at iol.unh.edu>
> *Sent:* Monday, January 12, 2026 11:28 PM
> *To:* Patrick Robb <probb at iol.unh.edu>; Akhil Goyal <gakhil at marvell.com>;
> fanzhang.oss at gmail.com
> *Cc:* luca.vizzarro at arm.com; dmarx at iol.unh.edu; dev at dpdk.org
> *Subject:* [EXTERNAL] Re: [RFC PATCH 3/3] dts: add cryptodev testsuite
>
>
>
> Hello, I am Andrew, a software test developer at UNH-IOL working on the
> DPDK Test Suite (DTS) component in DPDK. We're adding crypto device tests
> (both functional and performance) to DTS using the dpdk-test-crypto-perf
> application. These
>
> ZjQcmQRYFpfptBannerStart
>
> *Prioritize security for external emails: *
>
> Confirm sender and content safety before clicking links or opening
> attachments
>
>   *  Report Suspicious  *
> <https://us-phishalarm-ewt.proofpoint.com/EWT/v1/CRVmXkqW!uK3X-9Eb4hoauMMx8776juphaxuhWSpVPz0jRFbXZsENdFzDoFkHkisIFSRgpqYCKAsvJZ0J93ubWr88-ERDphsvU_UUPsR6-5g8RuDP-gdOzh7iwbOo1NwbQlg$>  ‌
>
>
> ZjQcmQRYFpfptBannerEnd
>
> Hello,
>
> I am Andrew, a software test developer at UNH-IOL working on the DPDK Test
> Suite (DTS) component in DPDK. We're adding crypto device tests (both
> functional and performance) to DTS using the dpdk-test-crypto-perf
> application. These are tests which can be run on real hardware, I.e. We
> have an Intel 8970 crypto card at the lab we are using for testing. I'm
> currently developing a performance test suite using option "--ptest
> throughput" that will fail tests when metrics deviate by n% from a
> baseline. I would like to get your feedback regarding how I can correctly
> check for performance regressions in the test cases by using
> dpdk-crypto-perf.
>
> 1. Which metrics in the attached table below are relevant for checking for
> performance regressions in throughput testing? MOps? Gbps?
> 2. What would be an acceptable throughput deviation between runs, and what
> percentage deviation should be considered a failure? For instance, right
> now I am thinking that if the test returns a Gbps reduction of 2% or
> greater from the given baseline, that will trigger a failure.
>
>
> Any guidance you can provide would be greatly appreciated.
> Best regards,
> Andrew
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mails.dpdk.org/archives/dev/attachments/20260121/1fd3f569/attachment-0001.htm>
-------------- next part --------------
A non-text attachment was scrubbed...
Name: image001.png
Type: image/png
Size: 122934 bytes
Desc: not available
URL: <http://mails.dpdk.org/archives/dev/attachments/20260121/1fd3f569/attachment-0001.png>