[PATCH v3 0/2] allow creating thread with real-time priority

[Morten Brørup]

> From: Thomas Monjalon [mailto:thomas at monjalon.net]
> Sent: Thursday, 26 October 2023 16.31
> 
> 26/10/2023 16:08, Morten Brørup:
> > > From: Thomas Monjalon [mailto:thomas at monjalon.net]
> > > Sent: Thursday, 26 October 2023 16.05
> > >
> > > 26/10/2023 15:57, Morten Brørup:
> > > > > From: Morten Brørup [mailto:mb at smartsharesystems.com]
> > > > > Sent: Thursday, 26 October 2023 15.45
> > > > >
> > > > > > From: Thomas Monjalon [mailto:thomas at monjalon.net]
> > > > > > Sent: Thursday, 26 October 2023 15.37
> > > > > >
> > > > > > 25/10/2023 18:31, Thomas Monjalon:
> > > > > > > Real-time thread priority was been forbidden on Unix
> > > > > > > because of problems they can cause.
> > > > > > > Warnings and helpers are added to avoid deadlocks,
> > > > > > > so real-time can be allowed on all systems.
> > > > > >
> > > > > > Unit test is failing:
> > > > > > DPDK:fast-tests / threads_autotest      TIMEOUT 600.01 s
> > > > > >
> > > > > > It is seen in only 1 target (maybe the failure occurence is random):
> > > > > >   Debian 11 (Buster) (ARM) | PASS
> > > > > >   Fedora 37 (ARM)          | PASS
> > > > > >   CentOS Stream 9 (ARM)    | FAIL
> > > > > >   Fedora 38 (ARM)          | PASS
> > > > > >   Fedora 38 (ARM Clang)    | PASS
> > > > > >   Ubuntu 20.04 (ARM)       | PASS
> > > > > >
> > > > > > I need to send a v4 with new implementation and better comments.
> > > > > > The Unix sleep will be upgraded from 1 ns to 1 us in case it makes a
> > > > > > difference.
> > > > >
> > > > > It will not make a difference. The kernel will go through the sleeping
> > > steps,
> > > > > then wake up again and see the real-time thread is ready to run, and
> then
> > > > > immediately schedule it.
> > > > >
> > > > > For testing purposes, consider sleeping 10 milliseconds or something
> > > > > significant like that.
> > > >
> > > > A bit more details...
> > > >
> > > > In our recent tests, nanosleep() itself took around 50 us. So you need
> to
> > > sleep longer than that for your thread not to be runnable when the
> nanosleep()
> > > wakes up again, because 50 us has already passed in "nanosleep overhead".
> > > > 10 milliseconds provides plenty of margin, and corresponds to 10 jiffies
> on
> > > a 1000 Hz kernel. (I don't know if it makes any difference for the kernel
> > > scheduler if the timer crosses a jiffy border or not.)
> > >
> > > 10 ms looks like an eternity.
> >
> > Agree. It is only for functional testing, not for production!
> 
> Realtime thread won't make any sense if we have to insert a long sleep.

It seems David came to our rescue here!

I have just tried running our test again with prctl(PR_SET_TIMERSLACK) of 1 ns, and the nanosleep(1 ns) delay dropped from ca. 50 us to ca. 2.5 us.

The timeout parameter to epoll_wait() is in milliseconds, which is useless for low-latency.
Perhaps real-time threads can be used with epoll() combined with timerfd for nanosecond resolution timeout.

> We need to find a good sleep value or give up with real-time threads.
> (note I'm not sure how much it is useful)

Only the application developer knows how much delay is acceptable. Which is why I mentioned that the new yield functions should document the delay.

> 
> > > I will try.
> > > (Anyway I did a mistake when sending v4)
> 
> I've sent a trial with 1 ms.
>