[dpdk-dev] [RFC PATCH v2 3/6] build: automatic NUMA and cpu counts detection

Juraj Linkeš juraj.linkes at pantheon.tech
Wed Oct 14 10:52:12 CEST 2020



> -----Original Message-----
> From: Bruce Richardson <bruce.richardson at intel.com>
> Sent: Wednesday, October 14, 2020 10:16 AM
> To: Juraj Linkeš <juraj.linkes at pantheon.tech>
> Cc: Ruifeng.Wang at arm.com; Honnappa.Nagarahalli at arm.com;
> Phil.Yang at arm.com; vcchunga at amazon.com; Dharmik.Thakkar at arm.com;
> jerinjacobk at gmail.com; hemant.agrawal at nxp.com; dev at dpdk.org
> Subject: Re: [RFC PATCH v2 3/6] build: automatic NUMA and cpu counts
> detection
> 
> On Wed, Oct 14, 2020 at 06:04:52AM +0000, Juraj Linkeš wrote:
> >
> >
> > > -----Original Message-----
> > > From: Bruce Richardson <bruce.richardson at intel.com>
> > > Sent: Tuesday, October 13, 2020 5:48 PM
> > > To: Juraj Linkeš <juraj.linkes at pantheon.tech>
> > > Cc: Ruifeng.Wang at arm.com; Honnappa.Nagarahalli at arm.com;
> > > Phil.Yang at arm.com; vcchunga at amazon.com; Dharmik.Thakkar at arm.com;
> > > jerinjacobk at gmail.com; hemant.agrawal at nxp.com; dev at dpdk.org
> > > Subject: Re: [RFC PATCH v2 3/6] build: automatic NUMA and cpu counts
> > > detection
> > >
> > > On Tue, Oct 13, 2020 at 04:54:39PM +0200, Juraj Linkeš wrote:
> > > > The build machine's number of cpus and numa nodes vary, resulting
> > > > in mismatched counts of RTE_MAX_LCORE and RTE_MAX_NUMA_NODES
> for
> > > many
> > > > builds. Automatically discover the host's numa and cpu counts to
> > > > remove this mismatch for native builds. Use current defaults for cross
> builds.
> > > > Leave users the option to override both if the specify a non-zero
> > > > amount on the command line.
> > > >
> > > > Signed-off-by: Juraj Linkeš <juraj.linkes at pantheon.tech>
> > > > ---
> > > >  buildtools/get_cpu_count.py  |  7 +++++++
> > > > buildtools/get_numa_count.py | 17 +++++++++++++++++
> > > >  buildtools/meson.build       |  2 ++
> > > >  config/meson.build           | 20 ++++++++++++++++++--
> > > >  meson_options.txt            |  8 ++++----
> > > >  5 files changed, 48 insertions(+), 6 deletions(-)  create mode
> > > > 100644 buildtools/get_cpu_count.py  create mode 100644
> > > > buildtools/get_numa_count.py
> > > >
> > > > diff --git a/buildtools/get_cpu_count.py
> > > > b/buildtools/get_cpu_count.py new file mode 100644 index
> > > > 000000000..386f85f8b
> > > > --- /dev/null
> > > > +++ b/buildtools/get_cpu_count.py
> > > > @@ -0,0 +1,7 @@
> > > > +#!/usr/bin/python3
> > > > +# SPDX-License-Identifier: BSD-3-Clause # Copyright (c) 2020
> > > > +PANTHEON.tech s.r.o.
> > > > +
> > > > +import os
> > > > +
> > > > +print(os.cpu_count())
> > > > diff --git a/buildtools/get_numa_count.py
> > > > b/buildtools/get_numa_count.py new file mode 100644 index
> > > > 000000000..f2ad35532
> > > > --- /dev/null
> > > > +++ b/buildtools/get_numa_count.py
> > > > @@ -0,0 +1,17 @@
> > > > +#!/usr/bin/python3
> > > > +# SPDX-License-Identifier: BSD-3-Clause # Copyright (c) 2020
> > > > +PANTHEON.tech s.r.o.
> > > > +
> > > > +import ctypes
> > > > +import glob
> > > > +import os
> > > > +
> > > > +if os.name == 'posix':
> > > > +    print(len(glob.glob('/sys/devices/system/node/node*')))
> > >
> > > FreeBSD also reports it's os.name as posix, but doesn't have a
> > > /sys/devices path to query.
> > >
> >
> > Do you know how do I get numa count on FreeBDS? I don't want to use the
> numa library if we can to avoid unneccesary dependencies. Do we need to cover
> some other cases other than FreeBDS, other Linux and Windows?
> >
> 
> To get numa count on freebsd, you can query it using "sysctl vm.ndomains"
> command. However, the bigger issue here is that you need something different
> other than os.name to distinguish between linux and freebsd.
> 

That should only be a matter of finding the right Python api - https://docs.python.org/3/library/sys.html?highlight=platform#sys.platform should be it.

> Regards,
> /Bruce



More information about the dev mailing list