[dpdk-dev] rte_mbuf.next in 2nd cacheline

[Ananyev, Konstantin]

> -----Original Message-----
> From: dev [mailto:dev-bounces at dpdk.org] On Behalf Of Bruce Richardson
> Sent: Monday, June 15, 2015 2:44 PM
> To: Olivier MATZ
> Cc: dev at dpdk.org; Damjan Marion (damarion)
> Subject: Re: [dpdk-dev] rte_mbuf.next in 2nd cacheline
> 
> On Mon, Jun 15, 2015 at 03:20:22PM +0200, Olivier MATZ wrote:
> > Hi Damjan,
> >
> > On 06/10/2015 11:47 PM, Damjan Marion (damarion) wrote:
> > >
> > > Hi,
> > >
> > > We noticed 7% performance improvement by simply moving rte_mbuf.next field to the 1st cache line.
> > >
> > > Currently, it falls under /* second cache line - fields only used in slow path or on TX */
> > > but it is actually used at several places in rx fast path. (e.g.: i40e_rx_alloc_bufs() is setting that field to NULL).
> > >
> > > Is there anything we can do here (stop using next field, or move it to 1st cache line)?
> >
> > Agree, this is also something I noticed, see:
> > http://dpdk.org/ml/archives/dev/2015-February/014400.html
> >
> > I did not have the time to do performance testing, but it's something
> > I'd like to do as soon as I can. I don't see any obvious reason not to
> > do it.
> >
> > It seems we currently just have enough room to do it (8 bytes are
> > remaining in the first cache line when compiled in 64 bits).
> 
> This, to me, is the obvious reason not to do it! It prevents us from taking in
> any other offload fields in the RX fast-path into the mbuf.
> 
> That being said, I can see why we might want to look to move it - but from the
> work done in the ixgbe driver, I'd be hopeful we can get as much performance with
> it on the second cache line for most cases, through judicious use of prefetching,
> or otherwise.
> 
> It took a lot of work and investigation to get free space in the mbuf - especially
> in cache line 0, and I'd like to avoid just filling the cache line up again as
> long as we possibly can!

Yep, agree with Bruce here.
Plus, with packet_type going to be 4B and vlan_tci_outer,
we just don't have 8 free bytes at the first cache line any more.
Konstantin

> 
> /Bruce
> 
> >
> >
> > Regards,
> > Olivier
> >
> >
> > >
> > > Thanks,
> > >
> > > Damjan
> > >
> > >
> > >
> > >