[dpdk-dev] [PATCH] eal: support using 0 as coremask for no-affinitization

[Van Haaren, Harry]

> -----Original Message-----
> From: Bruce Richardson <bruce.richardson at intel.com>
> Sent: Tuesday, February 16, 2021 5:31 PM
> To: Van Haaren, Harry <harry.van.haaren at intel.com>
> Cc: Burakov, Anatoly <anatoly.burakov at intel.com>; dev at dpdk.org
> Subject: Re: [dpdk-dev] [PATCH] eal: support using 0 as coremask for no-
> affinitization
> 
> On Tue, Feb 16, 2021 at 05:22:25PM +0000, Van Haaren, Harry wrote:
> > > -----Original Message-----
> > > From: Burakov, Anatoly <anatoly.burakov at intel.com>
> > > Sent: Tuesday, February 16, 2021 10:53 AM
> > > To: Richardson, Bruce <bruce.richardson at intel.com>; Van Haaren, Harry
> > > <harry.van.haaren at intel.com>
> > > Cc: dev at dpdk.org
> > > Subject: Re: [dpdk-dev] [PATCH] eal: support using 0 as coremask for no-
> > > affinitization
> > >
> > > On 16-Feb-21 10:46 AM, Bruce Richardson wrote:
> > > > On Tue, Feb 16, 2021 at 10:36:13AM +0000, Burakov, Anatoly wrote:
> > > >> On 16-Feb-21 9:43 AM, Bruce Richardson wrote:
> > > >>> Allow the user to specify that they don't want any core pinning from
> DPDK
> > > >>> by passing in the coremask of 0.
> > > >>> ---
> > > >>
> > > >> I haven't checked what happens yet, but down the line we also set affinity
> > > >> for service cores as well as interrupt thread. what would be the semantics
> > > >> of those in this particular case? do we want the same ability for service
> > > >> cores (i.e. pick a non-affinitized core)? And where does interrupt thread
> > > >> affinitize in this case (presumably, nowhere too)?
> > > >>
> > > > I have not checked the service core setup, because a) I forgot about them
> > > > and b) I'm not sure how their affinity rules work with respect to the main
> > > > lcore mask. On the other hand I did check out that the lcore mask for all
> > > > non-pinned threads, or control threads, is the full set of bits as
> > > > expected.
> > > >
> > > > /Bruce
> > > >
> > >
> > > +Harry,
> > >
> > > I believe service core mask must not overlap with lcore masks, so
> > > presumably using 0 as lcore mask would make it so that any service core
> > > mask will be valid (which is presumably what we want?).
> >
> > Services cores -S list or -s <mask> *must* overlap with the RTE lcores, EAL
> > then"steals" the service cores from the application lcores, code that
> implements here:
> > http://git.dpdk.org/dpdk-
> stable/tree/lib/librte_eal/common/eal_common_options.c?h=20.11#n657
> >
> > > Should service cores also have a "just pick a core" parameter?
> >
> > I'm not sure, depends on what the bigger goal is here.
> > Assuming we're enabling this for ROLE_RTE threads, then
> > it would seem to me that ROLE_SERVICE and control threads
> > would require similar treatment?
> >
> Control threads are affinitised to all cores not in the coremask, which
> means in this case that they can run anywhere on the system the OS chooses.

Ah ok, fair enough yes.

> In case of service cores, it would seem that using service cores with an
> empty coremask is just not compatible. I would assume that this
> incompatibility already exists when one has a coremask with only one core
> already in it.

Yes, correct, it would leave zero lcores for ROLE_RTE, meaning no lcores for the application.
A possible solution would be to special case a zero service core mask and apply the same
treatment as ROLE_RTE coremask?

Others likely have better ideas - I don't have time to follow DPDK threading/pinning topic
closely at the moment.