Large interruptions for EAL thread running on isol core

[Stephen Hemminger]

On Thu, 23 Jun 2022 21:03:49 +0200
Carsten Andrich <carsten.andrich at tu-ilmenau.de> wrote:

>  2. Use real-time priority (SCHED_FIFO w/ priority 99) for the DPDK
>     threads and
>     echo -1 > /proc/sys/kernel/sched_rt_runtime_us
>     to disable the runtime limit. With the runtime limit in place, the
>     SCHED_FIFO performance will be significantly worse than SCHED_OTHER.

This can cause major issues if application is normal DPDK application (never does system calls).
If an interrupt or other event happens on your isolated CPU, the work that it would
do in soft irq is never performed. FIFO has higher priority than kernel threads.
This can lead to mystery lockups from other applications (reads not completing, network timeouts, etc).

In any semi-recent kernel using SCHED_NO_HZ_FULL will keep clock ticks from happening
on the isolated cores.