[dpdk-users] dpdk and bulk data (video/audio)

Trahe, Fiona fiona.trahe at intel.com
Mon Sep 10 23:46:39 CEST 2018



> -----Original Message-----
> From: users [mailto:users-bounces at dpdk.org] On Behalf Of Wiles, Keith
> Sent: Monday, September 10, 2018 1:52 PM
> To: Sofia Baran <sofia.baran at gmx.net>
> Cc: users at dpdk.org
> Subject: Re: [dpdk-users] dpdk and bulk data (video/audio)
> 
> 
> 
> > On Sep 10, 2018, at 6:28 AM, Sofia Baran <sofia.baran at gmx.net> wrote:
> >
> >
> > Hi All,
> >
> > I want/try to us DPDK for transferring larger amount of data, e.g. video frames which usually are
> stored within memory buffers with sizes of several MB (remark: by using huges pages, these buffers
> could be physically contiguous).
> >
> > When looking at the DPDK documentation, library APIs and examples, I can't find a way/hint how to
> transfer larger buffers using DPDK without copying the video buffer fragments to the payload sections
> of the mbufs - which results in high CPU loads.
> >
> > Within the ip_fragmentation example indirect mbufs are used, pointing to the payload section of a
> direct mbuf (holding the header). But in my understanding the maximum size of a mbuf payload is 65KB
> (uint16_t)!?
> 
> It is true that mbufs only hold (64K - 1). The concept of mbufs is normally an ethernet packet and they
> are limited to 64K.
> 
> You can create a small mbuf (128 bytes) then set offset/data in the mbuf to point to the video buffer
> only if you can find the physical memory address for the data. The mbuf normally holds the physical
> address of the mbuf->data not the attached buffer in this case. This of course means you have to
> manage the mbuf internal structure members yourself and be very careful you do not rearrange the
> mbuf members as that can cause a performance problem.
> 

But the 64k-1 limit still applies, unless I'm misunderstanding.
A way to get around this is to use chained mbufs.
So create lots of small mbufs, each 128bytes, holding no payload, just the mbuf struct.
Chain them together with each buf_iova/buf_addr pointing to the next 64k-1 segment of the payload.
You'll need ~17 mbufs per MB, is this an acceptable overhead?
  
You can also consider using compressdev API to compress the data before transmitting.
However will have the same problem - and can use the same solution - for passing the data to the PMD to be compressed/decompressed.
 
> >
> > I'm pretty new to DPDK so maybe I missed something. I hope that someone can provide me some hits
> how to avoid copying the entire payload.
> >
> > Thanks
> > Sofia Baran
> >
> >
> 
> Regards,
> Keith



More information about the users mailing list