[dpdk-dev] Service lcores and Application lcores

[Thomas Monjalon]

30/06/2017 12:18, Van Haaren, Harry:
> From: Thomas Monjalon [mailto:thomas at monjalon.net]
> > 30/06/2017 10:52, Van Haaren, Harry:
> > > From: Thomas Monjalon [mailto:thomas at monjalon.net]
> > > > 29/06/2017 18:35, Van Haaren, Harry:
> > > > > 3) The problem;
> > > > >    If a service core runs the SW PMD schedule() function (option 2) *AND*
> > > > >    the application lcore runs schedule() func (option 1), the result is that
> > > > >    two threads are concurrently running a multi-thread unsafe function.
> > > >
> > > > Which function is multi-thread unsafe?
> > >
> > > With the current design, the service-callback does not have to be multi-thread safe.
> > > For example, the eventdev SW PMD is not multi-thread safe.
> > >
> > > The service library handles serializing access to the service-callback if multiple cores
> > > are mapped to that service. This keeps the atomic complexity in one place, and keeps
> > > services as light-weight to implement as possible.
> > >
> > > (We could consider forcing all service-callbacks to be multi-thread safe by using
> > atomics,
> > > but we would not be able to optimize away the atomic cmpset if it is not required. This
> > > feels heavy handed, and would cause useless atomic ops to execute.)
> > 
> > OK thank you for the detailed explanation.
> > 
> > > > Why the same function would be run by the service and by the scheduler?
> > >
> > > The same function can be run concurrently by the application, and a service core.
> > > The root cause that this could happen is that an application can *think* it is the
> > > only one running threads, but in reality one or more service-cores may be running
> > > in the background.
> > >
> > > The service lcores and application lcores existence without knowledge of the others
> > > behavior is the cause of concurrent running of the multi-thread unsafe service function.
> > 
> > That's the part I still don't understand.
> > Why an application would run a function on its own core if it is already
> > run as a service? Can we just have a check that the service API exists
> > and that the service is running?
> 
> The point is that really it is an application / service core mis-match.
> The application should never run a PMD that it knows also has a service core running it.

Yes

> However, porting applications to the service-core API has an over-lap time where an
> application on 17.05 will be required to call eg: rte_eventdev_schedule() itself, and
> depending on startup EAL flags for service-cores, it may-or-may-not have to call schedule() manually.

Yes service cores may be unavailable, depending of user configuration.
That's why it must be possible to request the service core API
to know whether a service is run or not.
When porting an application to service core, you just have to run this
check, which is known to be available for DPDK 17.08 (check rte_version.h).

> This is pretty error prone, and mis-configuration would cause A) deadlock due to no CPU cycles, B) segfault due to two cores.