[dpdk-dev] ***Spam*** Re: Mempool handler ops index allocation issue

[Eads, Gage]

> -----Original Message-----
> From: Andrew Rybchenko [mailto:arybchenko at solarflare.com]
> Sent: Monday, May 13, 2019 7:22 AM
> To: Olivier Matz <olivier.matz at 6wind.com>; Eads, Gage
> <gage.eads at intel.com>
> Cc: dev at dpdk.org
> Subject: Re: ***Spam*** Re: Mempool handler ops index allocation issue
> 
> Hi Olivier,
> 
> On 5/13/19 3:14 PM, Olivier Matz wrote:
> > Hi Gage,
> >
> > On Thu, May 09, 2019 at 10:19:55PM +0000, Eads, Gage wrote:
> >> Hi all,
> >>
> >> I ran into a problem with a multi-process application, in which two
> processes assigned the same mempool handler ops index to *different*
> handlers. This happened because the two processes supplied the -d EAL
> arguments in different order, e.g.:
> >>
> >> sudo ./appA -dlibrte_mempool_bucket.so -dlibrte_mempool_ring.so
> >> --proc-type primary & sudo ./appB -dlibrte_mempool_ring.so
> >> -dlibrte_mempool_bucket.so --proc-type secondary &
> >>
> >> The dynamic load order matters because the ops indexes are assigned in
> the order the library ctors are run. This can result in the different processes
> trying to use different handlers for the same mempool.
> >>
> >> I'm not aware of any requirement that the EAL argument order should
> match across processes, so I don't think this is a user error. This could also
> happen in static libraries if they linked the libraries in a different order - but
> that shouldn't occur if both applications are following the rules for building an
> external application
> (https://doc.dpdk.org/guides/prog_guide/dev_kit_build_system.html#build
> ing-external-applications).
> >>
> >> If you agree that this is an issue, I see a couple possible resolutions:
> >>
> >>
> >> 1.       Add a note/warning to the mempool handlers section of the user
> guide
> (https://doc.dpdk.org/guides/prog_guide/mempool_lib.html#mempool-
> handlers)
> >>
> >> 2.       Modify rte_mempool_register_ops() so that built-in handlers (ring,
> stack, etc.) have fixed IDs. E.g. ring is always 0, stack is always 1, etc. These
> handlers could be identified by their name string. User-registered mempools
> would still be susceptible to this problem, though.
> >>
> >> Thoughts? Alternatives?
> > What about this:
> >
> > - add a new table in a named memory zone that stores the association
> >    between mempool_ops id and name (but not the ops pointers) of the
> >    primary process.
> >
> > - change rte_mempool_register_ops() to have a specific behavior in case
> >    of a secondary process: lookup in that table to get the id associated
> >    to the name (fail if not found).
> >

Sorry for the delay, just revisiting this now.

Memory zones won't be available in rte_mempool_register_ops when using static libraries (or shared libraries loaded at program startup), since the ctors execute before rte_eal_init() is called.

> >
> > On the other hand, using secondary processes always looked a bit
> > dangerous to me (for several reasons), so adding a note in the
> > programmer's guide (your proposal 1) is also fine to me.

Considering the memzone issue, I'll go this route.

Thanks,
Gage