[dpdk-users] Linux descheduling DPDK transmit thread in SMP system? Any help greatly appreciated...

[Stephen Hemminger]

On Tue, 5 Sep 2017 17:35:16 +0100 (BST)
"terry.montague.1980 at btinternet.com" <terry.montague.1980 at btinternet.com> wrote:

> Hi all,
> I'm running a DPDK transmit thread with the ixgbe PMD on Ubuntu 17.04, with dual Xeon E5-2623 v4 CPUs  on an Intel S2600 motherboard. The cores in use by DPDK are isolated in the boot cmdline with "isolcpus"
> What I'm observing is I believe, on occasion, the DPDK transmit thread being descheduled by the system for between 20 and 60 milliseconds.
> I've tried running as Real time with max priority, and tweaking the scheduling through /proc/sys/kernel/sched_rt_runtime_us to be 99.999% available for real time thread, but so far no improvement
> I notice there are various kernel threads resident on the isolated cpus, cpuhp/watchdog/migration/ksoftirqd +multiple kworker threads.
> I can only assume the system is occasionally running these kernel side instead, although the CPU time taken  (tens of milliseconds) seems very high. Note please that I amd NOT running with a modified kernel (nohz_full is not set).
> Does anyone have any advice to keep the DPDK transmit thread running for more of the time on this SMP system ?
> Many thanks
> Terry.

Probably not a Linux scheduler issue. More likely some system hardware thing like SMT or ME.