[dpdk-dev] [PATCH v2] build: optional NUMA and cpu counts detection

Juraj Linkeš juraj.linkes at pantheon.tech
Fri Apr 16 14:43:39 CEST 2021


I didn't notice this e-mail, since it was sent just to dev at dpdk.org and not to me explicitly, which left it unnoticed by my filters, unfortunately.

> -----Original Message-----
> From: dev <dev-bounces at dpdk.org> On Behalf Of David Christensen
> Sent: Wednesday, March 31, 2021 9:07 PM
> To: dev at dpdk.org
> Subject: Re: [dpdk-dev] [PATCH v2] build: optional NUMA and cpu counts
> detection
> 
> 
> 
> On 3/31/21 2:06 AM, Juraj Linkeš wrote:
> > Add an option to automatically discover the host's numa and cpu counts
> > and use those values for a non cross-build.
> > Give users the option to override the per-arch default values or
> > values from cross files by specifying them on the command line with
> > -Dmax_lcores and -Dmax_numa_nodes.
> >
> > Signed-off-by: Juraj Linkeš <juraj.linkes at pantheon.tech>
> > Reviewed-by: Honnappa Nagarahalli <honnappa.nagarahalli at arm.com>
> > ---
> >   MAINTAINERS                  |  2 ++
> >   buildtools/get-cpu-count.py  |  7 ++++++
> >   buildtools/get-numa-count.py | 22 +++++++++++++++++
> >   buildtools/meson.build       |  2 ++
> >   config/meson.build           | 47 ++++++++++++++++++++++++++++++++++--
> >   config/x86/meson.build       |  2 ++
> >   meson_options.txt            |  8 +++---
> >   7 files changed, 84 insertions(+), 6 deletions(-)
> >   create mode 100644 buildtools/get-cpu-count.py
> >   create mode 100644 buildtools/get-numa-count.py
> >
> > diff --git a/MAINTAINERS b/MAINTAINERS index 0ec5588540..7270f33cf5
> > 100644
> > --- a/MAINTAINERS
> > +++ b/MAINTAINERS
> > @@ -99,6 +99,8 @@ F: meson_options.txt
> >   F: config/
> >   F: buildtools/chkincs/
> >   F: buildtools/call-sphinx-build.py
> > +F: buildtools/get-cpu-count.py
> > +F: buildtools/get-numa-count.py
> >   F: buildtools/list-dir-globs.py
> >   F: buildtools/pkg-config/
> >   F: buildtools/symlink-drivers-solibs.sh
> > diff --git a/buildtools/get-cpu-count.py b/buildtools/get-cpu-count.py
> > new file mode 100644 index 0000000000..317b32088f
> > --- /dev/null
> > +++ b/buildtools/get-cpu-count.py
> > @@ -0,0 +1,7 @@
> > +#!/usr/bin/env python3
> > +# SPDX-License-Identifier: BSD-3-Clause # Copyright (c) 2021
> > +PANTHEON.tech s.r.o.
> > +
> > +import os
> > +
> > +print(os.cpu_count())
> 
> Returns the expected value on a P9 system.
> 
> > diff --git a/buildtools/get-numa-count.py
> > b/buildtools/get-numa-count.py new file mode 100644 index
> > 0000000000..77ef2b9f24
> > --- /dev/null
> > +++ b/buildtools/get-numa-count.py
> > @@ -0,0 +1,22 @@
> > +#!/usr/bin/env python3
> > +# SPDX-License-Identifier: BSD-3-Clause # Copyright (c) 2021
> > +PANTHEON.tech s.r.o.
> > +
> > +import ctypes
> > +import glob
> > +import os
> > +import subprocess
> > +
> > +if os.name == 'posix':
> > +    if os.path.isdir('/sys/devices/system/node'):
> > +        print(len(glob.glob('/sys/devices/system/node/node*')))
> > +    else:
> > +        subprocess.run(['sysctl', '-n', 'vm.ndomains'])
> > +
> > +elif os.name == 'nt':
> > +    libkernel32 = ctypes.windll.kernel32
> > +
> > +    count = ctypes.c_ulong()
> > +
> > +    libkernel32.GetNumaHighestNodeNumber(ctypes.pointer(count))
> > +    print(count.value + 1)
> 
> Does not return the expected value on my P9 system (NUMA nodes are not
> contiguous).  Got 6, expect to see at least 8 otherwise I can't use lcores on
> NUMA node 8.
> 

I guess the problem here is that we're not really setting the number of NUMA nodes that are usable, but the highest NUMA that is usable?

In that case, the proper config in should be 256 so that even node255 is usable, is that right?

> $ python3 ./get-numa-count.py
> 6
> $ lscpu
> Architecture:        ppc64le
> Byte Order:          Little Endian
> CPU(s):              128
> On-line CPU(s) list: 0-127
> Thread(s) per core:  4
> Core(s) per socket:  16
> Socket(s):           2
> NUMA node(s):        6
> Model:               2.3 (pvr 004e 1203)
> Model name:          POWER9, altivec supported
> CPU max MHz:         3800.0000
> CPU min MHz:         2300.0000
> L1d cache:           32K
> L1i cache:           32K
> L2 cache:            512K
> L3 cache:            10240K
> NUMA node0 CPU(s):   0-63
> NUMA node8 CPU(s):   64-127
> NUMA node252 CPU(s):
> NUMA node253 CPU(s):
> NUMA node254 CPU(s):
> NUMA node255 CPU(s):
> 
> See attached ls-node.txt.gz for full directory structure on a P9 system.
> 
> Dave


More information about the dev mailing list