[dpdk-users] Run-to-completion or Pipe-line for QAT PMD in DPDK

Changchun Zhang changchun.zhang at oracle.com
Fri Jan 18 18:55:58 CET 2019



Thanks!
Changchun (Alex)


-----Original Message-----
From: Trahe, Fiona [mailto:fiona.trahe at intel.com] 
Sent: Friday, January 18, 2019 11:58 AM
To: Changchun Zhang <changchun.zhang at oracle.com>; Pathak, Pravin <pravin.pathak at intel.com>; users at dpdk.org
Cc: Trahe, Fiona <fiona.trahe at intel.com>
Subject: RE: [dpdk-users] Run-to-completion or Pipe-line for QAT PMD in DPDK

Hi Alex,

> -----Original Message-----
> From: Changchun Zhang [mailto:changchun.zhang at oracle.com]
> Sent: Friday, January 18, 2019 4:42 PM
> To: Trahe, Fiona <fiona.trahe at intel.com>; Pathak, Pravin 
> <pravin.pathak at intel.com>; users at dpdk.org
> Subject: RE: [dpdk-users] Run-to-completion or Pipe-line for QAT PMD 
> in DPDK
> 
> 
> 
> Thanks!
> Changchun (Alex)
> 
> 
> -----Original Message-----
> From: Trahe, Fiona [mailto:fiona.trahe at intel.com]
> Sent: Friday, January 18, 2019 11:26 AM
> To: Changchun Zhang <changchun.zhang at oracle.com>; Pathak, Pravin 
> <pravin.pathak at intel.com>; users at dpdk.org
> Cc: Trahe, Fiona <fiona.trahe at intel.com>
> Subject: RE: [dpdk-users] Run-to-completion or Pipe-line for QAT PMD 
> in DPDK
> 
> Hi Alex,
> > [changchun] In the same thread, but how about to dequeuer at the 
> > beginning of the thread each time, if data presents then processing 
> > them, if no data just do other work, and equeue the packets at some time but does not wait.
> > For example:
> > While(1)
> > {
> > 	Nb_ops = dequeuer();
> > 	If(nb_ops > )
> >              {
> >                  process_dequeued_data();
> >                  continue;
> >              }
> >
> >               Other_work();
> >               If(ipsec)
> >                   Enqueuer();
> > }
> > Does it make sense?
> [Fiona] It can, though on the first loop ro after a queit time youll 
> proably get very few back on first and second dequeue as It'll be 
> called immediately after the enqueue. Once it gets busy that could be 
> ok though [changchun] Thank you Fiona. One more question, as you said, 
> enqueuer/dequeue should be called within the same thread. Why? Is it 
> because the other thread(lcore 1) cannot dequeuer the processed data 
> from other thread(lcore 2)? But as the cryptograph device lib doc 
> says, "it is howerver possible to use a different logical core to dequeuer an operation on a queue pair from the logical core which it was enqueued on". Looking forward to more details.
It's because the QAT inflight counter would be incremented and decremented by both threads so would need to be an atomic. It used to be atomic until 17.11 release but we got a good reduction in offload cycle-count by replacing this with a normal variable and as all the feedback we got was that applications were not using in pipeline mode we decided to trade off this limitation for the added performance. The limitation is documented here:
https://urldefense.proofpoint.com/v2/url?u=http-3A__doc.dpdk.org_guides_cryptodevs_qat.html&d=DwIFAg&c=RoP1YumCXCgaWHvlZYR8PZh8Bv7qIrMUB65eapI_JnE&r=VCO7jqQFACS5pfBnqRHRln4bZ3htFERYZ3PkX0ytpns&m=Bp2X1mxAJ_Eg1kB1t91mgFDUfIrr8Vl-2Nxq_9RLxrI&s=2MPxBDAAKTRMsgE6mrw0cbGX8C4oFcFstJWzpEp8opI&e=
You can look at code before 17.11 release to see the difference.

 [changchun] Many thanks! So from this limitation, we can conclude that Lcore can only dequeue the QAT queue which was enqueued by itself, right. If so, then the Crypto device lib doc may be a little misleading, at least some notes should be put there.




More information about the users mailing list