[dpdk-dev] [RFC 0/8] mbuf: structure reorganization

[Bruce Richardson]

On Tue, Feb 21, 2017 at 05:12:12PM +0100, Jan Blunck wrote:
> On Tue, Feb 21, 2017 at 10:54 AM, Olivier MATZ <olivier.matz at 6wind.com> wrote:
> > Hi Jan,
> >
> > On Mon, 20 Feb 2017 10:27:40 +0100, Jan Blunck <jblunck at infradead.org>
> > wrote:
> >> On Fri, Feb 17, 2017 at 3:17 PM, Olivier Matz
> >> <olivier.matz at 6wind.com> wrote:
> >> >
> >> > Sorry, I don't really get your point. My comprehension of the
> >> > timestamp usage in a PMD is as following:
> >> >
> >> > rx_burst(struct rxq *rxq, ...)
> >> > {
> >> >         unsigned long factor = rxq->timestamp_factor;
> >> >         unsigned port = rxq->port;
> >> >
> >> >         for each hw_desc {
> >> >                 m = rte_pktmbuf_alloc(rxq->pool);
> >> >                 m->len = hw_desc->len;
> >> >                 m->port = port;
> >> >                 m->ol_flags =
> >> >                 ...
> >> >                 m->timestamp = hw_desc->timestamp * factor;
> >> >         }
> >> >         ...
> >> > }
> >> >
> >> > In that case, I think it deserves to be in the 1st cache line.
> >> >
> >>
> >> Timestamps are non-functional data. I believe they don't deserve to be
> >> data normalized.
> >
> > I don't really see why timestamp is non-functional, compared to rss
> > hash, flow director id, packet type, seqn, ...
> >
> 
> One doesn't drop, forward, reorder or decrypt packets based on the
> timestamp value.
> 
> Common use cases for timestamps are analytics applications or circuit
> breakers where you want to ensure that your algos don't act upon stale
> data. This is more application layer / slow path work.
> 
> > I think the goal of DPDK is to provide a generic/normalized API for
> > such features (like for offload), so it can be used by an application
> > whatever the underlying driver.
> >
> 
> I believe there is a difference between dataplane relevant
> functionally and non-functional features. If you structurally
> normalize the later you will end up making everyone pay for a feature
> that is only relevant to a particular group of users. In that case I
> would at least expect that the support for timestamps can get selected
> at compile time.
> 
> >>
> >> It would be beneficial to normalize the access of non-functional data.
> >> That would require some kind of extension or feature flags, e.g. like
> >> some uverb structures support extensions. This would allow the NICs
> >> that support timestamps to do the minimal amount of work during burst
> >> and defer data normalization to the time of access. That would also
> >> safe everyone from wasting 64bit in the first cacheline in case
> >> timestamps are not supported or unused.
> >
> > I agree that we could start to think about "extensible" mbufs. In few
> > words, I think we could have:
> > - only general fields are static
> > - the application registers room in mbuf structure for all features it
> >   will use
> > It would rationalize the room used in mbuf, at the cost of a more
> > complex/slow access to the data in PMDs and application, since the
> > offset won't be static.
> >
> 
> Access through PMD specific function pointers should be relatively
> fast on access. Modern architecture optimize that use case well
> enough.
>
The cost of doing a function call per packet to access data starts to
add up very, very fast. For the app, once the data is written to the
mbuf, it should be in the L1 cache, giving very fast access to it in a
few cycles. However, if a function call has to be made in order to do
the read, that makes the read of that field many times more expensive.

/Bruce