[dpdk-users] lcore stack location and NUMA nodes

N. Benes nbenes at eso.org
Tue May 14 16:50:00 CEST 2019


Hi all,

is it ensured that the stacks of lcore threads with set affinity are
located on the same NUMA node as the assigned CPU core?

So far, I understand that e.g. `rte_eal_mp_remote_launch()` does not
actually create threads but just communicates the function to execute to
already existing threads (via pipes). These threads are created during
initialisation of the EAL:

https://git.dpdk.org/dpdk/tree/lib/librte_eal/linux/eal/eal.c#n1164

The threads execute a state machine `eal_thread_loop()` where the
affinity is changed/set for the corresponding lcore/thread:

https://git.dpdk.org/dpdk/tree/lib/librte_eal/linux/eal/eal_thread.c#n119

However, couldn't this lead to the stacks of some threads to be still
located on the (different) NUMA node where `pthread_create()` was called
from, i.e. possible decreased performance?
I did not find an explicit relocation of thread stacks.
(I also did not find anything related in the glibc/nptl code)

One possible fix may be to change the affinity of the main/master thread
before calling `pthread_create()` for the slaves and to reset the
affinity of the master thread after all slave threads are created.

Cheers!


More information about the users mailing list