[dpdk-dev] [PATCH 1/6] cxgbe: Optimize forwarding performance for 40G

[Ananyev, Konstantin]

> -----Original Message-----
> From: dev [mailto:dev-bounces at dpdk.org] On Behalf Of Rahul Lakkireddy
> Sent: Monday, October 05, 2015 11:06 AM
> To: Aaron Conole
> Cc: dev at dpdk.org; Felix Marti; Kumar A S; Nirranjan Kirubaharan
> Subject: Re: [dpdk-dev] [PATCH 1/6] cxgbe: Optimize forwarding performance for 40G
> 
> Hi Aaron,
> 
> On Friday, October 10/02/15, 2015 at 14:48:28 -0700, Aaron Conole wrote:
> > Hi Rahul,
> >
> > Rahul Lakkireddy <rahul.lakkireddy at chelsio.com> writes:
> >
> > > Update sge initialization with respect to free-list manager configuration
> > > and ingress arbiter. Also update refill logic to refill mbufs only after
> > > a certain threshold for rx.  Optimize tx packet prefetch and free.
> > <<snip>>
> > >  			for (i = 0; i < sd->coalesce.idx; i++) {
> > > -				rte_pktmbuf_free(sd->coalesce.mbuf[i]);
> > > +				struct rte_mbuf *tmp = sd->coalesce.mbuf[i];
> > > +
> > > +				do {
> > > +					struct rte_mbuf *next = tmp->next;
> > > +
> > > +					rte_pktmbuf_free_seg(tmp);
> > > +					tmp = next;
> > > +				} while (tmp);
> > >  				sd->coalesce.mbuf[i] = NULL;
> > Pardon my ignorance here, but rte_pktmbuf_free does this work. I can't
> > actually see much difference between your rewrite of this block, and
> > the implementation of rte_pktmbuf_free() (apart from moving your branch
> > to the end of the function). Did your microbenchmarking really show this
> > as an improvement?
> >
> > Thanks for your time,
> > Aaron
> 
> rte_pktmbuf_free calls rte_mbuf_sanity_check which does a lot of
> checks. 

Only when RTE_LIBRTE_MBUF_DEBUG is enabled in your config.
By default it is switched off. 

> This additional check seems redundant for single segment
> packets since rte_pktmbuf_free_seg also performs rte_mbuf_sanity_check.
> 
> Several PMDs already prefer to use rte_pktmbuf_free_seg directly over
> rte_pktmbuf_free as it is faster.

Other PMDs use rte_pktmbuf_free_seg() as each TD has an associated 
with it segment. So as HW is done with the TD, SW frees associated segment.
In your case I don't see any point in re-implementing rte_pktmbuf_free() manually,
and I don't think it would be any faster.

Konstantin  

> 
> The forwarding perf. improvement with only this particular block is
> around 1 Mpps for 64B packets when using l3fwd with 8 queues.
> 
> Thanks,
> Rahul