[dpdk-dev] [PATCH v3 6/9] eal: register non-EAL threads as lcores

[Ananyev, Konstantin]

> -----Original Message-----
> From: David Marchand <david.marchand at redhat.com>
> Sent: Wednesday, June 24, 2020 10:24 AM
> To: Ananyev, Konstantin <konstantin.ananyev at intel.com>
> Cc: dev at dpdk.org; jerinjacobk at gmail.com; Richardson, Bruce <bruce.richardson at intel.com>; mdr at ashroe.eu; ktraynor at redhat.com;
> Stokes, Ian <ian.stokes at intel.com>; i.maximets at ovn.org; Thomas Monjalon <thomas at monjalon.net>; Mcnamara, John
> <john.mcnamara at intel.com>; Kovacevic, Marko <marko.kovacevic at intel.com>; Burakov, Anatoly <anatoly.burakov at intel.com>; Olivier
> Matz <olivier.matz at 6wind.com>; Andrew Rybchenko <arybchenko at solarflare.com>; Neil Horman <nhorman at tuxdriver.com>
> Subject: Re: [dpdk-dev] [PATCH v3 6/9] eal: register non-EAL threads as lcores
> 
> On Tue, Jun 23, 2020 at 3:16 PM Ananyev, Konstantin
> <konstantin.ananyev at intel.com> wrote:
> > > Even before this series, MP has no protection on lcore placing between
> > > primary and secondary processes.
> >
> > Agree, it is not a new problem, it has been there for a while.
> > Though making lcore assignment dynamic will make it more noticeable and harder to avoid.
> > With static only lcore distribution it is much easier to control things.
> >
> > > Personally, I have no use for DPDK MP and marking MP as not supporting
> > > this new feature is tempting for a first phase.
> > > If this is a strong requirement, I can look at it in a second phase.
> > > What do you think?
> >
> > In theory it is possible to mark this new API as not supported for MP.
> > At least for now. Though I think it is sort of temporal solution.
> > AFAIK, MP is used by customers, so sooner or later someone will hit that problem.
> 
> I understand this argument.
> But then we don't see those customers giving feedback.
> 
> 
> > Let say, we do have pdump app/library in our mainline.
> > As I can see - it will be affected when users will start using this new dynamic lcore API
> > inside their apps.
> 
> Supporting lcore allocation in MP requires exchanges between
> primary/secondary processes like what we have for memory allocations.
> It will be quite a beast to get to work fine, while not even knowing
> if people actually want to use both.

I don't think we need to re-implement RPC as we did for memory subsystem.
One relatively simple approach - move lcore_role[] and related lock into
shared memory (separate memzone or so).
I think it should help a lot and will solve majority of the problems.
One limitation - init/fini callbacks can be static only.
As the drawback, it will introduce change in current behaviour:
secondary process with lcore-mask that intersects with master lcore-mask
will fail to start.
Second approach - make lcore_id local process entity:
prohibit indexing by lcore_id in shared data structures.
Let say for mempool - make cache local (per process).
While that approach is probably more elegant and consistent,
it would require more work and will cause ABI (maybe API also) breakage.
 
> For v4, I added a check to exclude MP and the new API.

Do you mean - make this new dynamic-lcore API return an error if callied
from secondary process?

> I am still willing to help if people do care about using both features together.