[dpdk-dev] [RFC PATCH v2 3/6] build: automatic NUMA and cpu counts detection

[Bruce Richardson]

On Wed, Oct 14, 2020 at 06:04:52AM +0000, Juraj Linkeš wrote:
> 
> 
> > -----Original Message-----
> > From: Bruce Richardson <bruce.richardson at intel.com>
> > Sent: Tuesday, October 13, 2020 5:48 PM
> > To: Juraj Linkeš <juraj.linkes at pantheon.tech>
> > Cc: Ruifeng.Wang at arm.com; Honnappa.Nagarahalli at arm.com;
> > Phil.Yang at arm.com; vcchunga at amazon.com; Dharmik.Thakkar at arm.com;
> > jerinjacobk at gmail.com; hemant.agrawal at nxp.com; dev at dpdk.org
> > Subject: Re: [RFC PATCH v2 3/6] build: automatic NUMA and cpu counts
> > detection
> > 
> > On Tue, Oct 13, 2020 at 04:54:39PM +0200, Juraj Linkeš wrote:
> > > The build machine's number of cpus and numa nodes vary, resulting in
> > > mismatched counts of RTE_MAX_LCORE and RTE_MAX_NUMA_NODES for
> > many
> > > builds. Automatically discover the host's numa and cpu counts to
> > > remove this mismatch for native builds. Use current defaults for cross builds.
> > > Leave users the option to override both if the specify a non-zero
> > > amount on the command line.
> > >
> > > Signed-off-by: Juraj Linkeš <juraj.linkes at pantheon.tech>
> > > ---
> > >  buildtools/get_cpu_count.py  |  7 +++++++
> > > buildtools/get_numa_count.py | 17 +++++++++++++++++
> > >  buildtools/meson.build       |  2 ++
> > >  config/meson.build           | 20 ++++++++++++++++++--
> > >  meson_options.txt            |  8 ++++----
> > >  5 files changed, 48 insertions(+), 6 deletions(-)  create mode 100644
> > > buildtools/get_cpu_count.py  create mode 100644
> > > buildtools/get_numa_count.py
> > >
> > > diff --git a/buildtools/get_cpu_count.py b/buildtools/get_cpu_count.py
> > > new file mode 100644 index 000000000..386f85f8b
> > > --- /dev/null
> > > +++ b/buildtools/get_cpu_count.py
> > > @@ -0,0 +1,7 @@
> > > +#!/usr/bin/python3
> > > +# SPDX-License-Identifier: BSD-3-Clause # Copyright (c) 2020
> > > +PANTHEON.tech s.r.o.
> > > +
> > > +import os
> > > +
> > > +print(os.cpu_count())
> > > diff --git a/buildtools/get_numa_count.py
> > > b/buildtools/get_numa_count.py new file mode 100644 index
> > > 000000000..f2ad35532
> > > --- /dev/null
> > > +++ b/buildtools/get_numa_count.py
> > > @@ -0,0 +1,17 @@
> > > +#!/usr/bin/python3
> > > +# SPDX-License-Identifier: BSD-3-Clause # Copyright (c) 2020
> > > +PANTHEON.tech s.r.o.
> > > +
> > > +import ctypes
> > > +import glob
> > > +import os
> > > +
> > > +if os.name == 'posix':
> > > +    print(len(glob.glob('/sys/devices/system/node/node*')))
> > 
> > FreeBSD also reports it's os.name as posix, but doesn't have a /sys/devices path
> > to query.
> > 
> 
> Do you know how do I get numa count on FreeBDS? I don't want to use the numa library if we can to avoid unneccesary dependencies. Do we need to cover some other cases other than FreeBDS, other Linux and Windows?
> 

To get numa count on freebsd, you can query it using "sysctl vm.ndomains"
command. However, the bigger issue here is that you need something
different other than os.name to distinguish between linux and freebsd.

Regards,
/Bruce