[EXTERNAL] Re: [dpdk-dev] [PATCH v16 2/9] eal: add thread attributes

[Tyler Retzlaff]

-----Original Message-----
From: Bruce Richardson <bruce.richardson at intel.com> 
Sent: Wednesday, November 10, 2021 1:21 AM
To: Stephen Hemminger <stephen at networkplumber.org>
Cc: Tyler Retzlaff <roretzla at microsoft.com>; Narcisa Ana Maria Vasile <navasile at linux.microsoft.com>; thomas <thomas at monjalon.net>; dev at dpdk.org; Dmitry Kozlyuk <dmitry.kozliuk at gmail.com>; Khoa To <khot at microsoft.com>; Dmitry Malloy <dmitrym at microsoft.com>; talshn <talshn at mellanox.com>; Omar Cardona <ocardona at microsoft.com>; david.marchand at redhat.com; Kadam, Pallavi <pallavi.kadam at intel.com>
Subject: Re: [EXTERNAL] Re: [dpdk-dev] [PATCH v16 2/9] eal: add thread attributes

> On Tue, Nov 09, 2021 at 10:29:25PM -0800, Stephen Hemminger wrote:
> > On Wed, 10 Nov 2021 04:20:52 +0000
> > Tyler Retzlaff <roretzla at microsoft.com> wrote:
> > 
> > > > >
> > > > > High performance applications benefit from an option to raise 
> > > > > the priority of their threads to avoid being preemted by other 
> > > > > threads on the system. If there are issues with realtime 
> > > > > priority on some of the platforms, maybe we can add a warning 
> > > > > for the user to make them aware of possible crashes as Stephen 
> > > > > H. suggested some time ago. Note that this patch doesn't change the priority of EAL threads, enabling the higher priority will be done through a command line option when starting the application.
> > > > > Maybe we can explore raising the priority but not to the realtime level.  
> > > 
> > > > Let me put it more succulently.
> > > > Almost all DPDK applications have threads that are a 100% CPU doing polling.
> > > > Putting those thread as real-time thread breaks Linux badly because the kernel can and will try and run work on those CPU's and the system is broken/unstable/dead at that point.  
> > > 
> > > The suggestion is that when running on Windows it should be possible for the application to be configured to use real-time threads, there is no implication that it will force real-time priority on Linux.  If it doesn't make sense for it to be configured on Linux then don't configure it.  But saying it shall not be configurable for any platform just because one platform can't make use of the configuration and those platforms have to run compromised makes little sense.  Linux administrators are (or should be) knowledgeable enough to know what configuration to use.
> > >
> > 
> > Why not just make it always return an error when real-time is requested on Linux from a DPDK thread.
> >
> I would tend to agree with this suggestion, that we just return ENOTSUP error code for Linux. It's too easy to lock up a linux system with realtime DPDK threads. If a user really knows what they are doing and they need realtime priority on linux, they can use the linux scheduler APIs directly.

This sounds reasonable to me.  It doesn't compromise the Linux platform and allows the Windows platform to reach potential.

> However, I am ok with having such a function supported on Windows if it's needed. Unfortunately, I don't know for sure about BSD, but it probably should follow the linux approach initially unless proved safe.

Agreed, probably true for all POSIX platforms that support RT extensions.