[dpdk-dev] [PATCH v2] usertools: add huge page setup script

[Bruce Richardson]

On Fri, Sep 04, 2020 at 03:58:03PM +0100, Burakov, Anatoly wrote:
> On 03-Sep-20 11:48 PM, Stephen Hemminger wrote:
> > This is an improved version of the setup of huge pages
> > bases on earlier DPDK setup. Differences are:
> >     * it autodetects NUMA vs non NUMA
> >     * it allows setting different page sizes
> >       recent kernels support multiple sizes.
> >     * it accepts a parameter in bytes (not pages).
> > 
> > If necessary the steps of clearing old settings and mounting/umounting
> > can be done individually.
> > 
> > 
> > Signed-off-by: Stephen Hemminger <stephen at networkplumber.org>
> > ---
> > v2 -- rewrite in python
> > 	The script is python3 only because supporting older versions
> > 	no longer makes any sense.
> > 
> >   usertools/hugepage-setup.py | 317 ++++++++++++++++++++++++++++++++++++
> >   1 file changed, 317 insertions(+)
> >   create mode 100644 usertools/hugepage-setup.py
> > 
> > diff --git a/usertools/hugepage-setup.py b/usertools/hugepage-setup.py
> > new file mode 100644
> > index 000000000000..8e7642428d9e
> > --- /dev/null
> > +++ b/usertools/hugepage-setup.py
> > @@ -0,0 +1,317 @@
> > +# Copyright (c) 2020 Microsoft Corporation
> > +#
> > +# Script to query and setup huge pages for DPDK applications.
> > +
> > +import sys
> > +import os
> > +import re
> > +import getopt
> > +import glob
> > +from os.path import exists, basename
> > +
> > +# convention for where to mount huge pages
> > +hugedir = '/dev/hugepages'
> 
> This isn't a "convention", this is a default systemd mountpoint.
> 
> > +
> > +# command-line flags
> > +show_flag = None
> > +reserve_kb = None
> > +clear_flag = None
> > +hugepagesize_kb = None
> > +mount_flag = None
> > +unmount_flag = None
> > +
> > +
> > +def usage():
> > +    '''Print usage information for the program'''
> > +    global hugedir
> > +    mnt = hugedir
> > +    argv0 = basename(sys.argv[0])
> > +    print("""
> > +Usage:
> > +------
> > +    %(argv0)s [options]
> > +
> > +Options:
> > +    --help, --usage:
> > +        Display usage information and quit
> > +
> > +    -s, --show:
> > +        Print the current huge page configuration.
> > +
> > +    --setup:
> > +        Simplified version of clear, umount, reserve, mount operations
> > +
> > +    -c, --clear:
> > +        Remove all huge pages
> > +
> > +    -r, --reserve:
> > +        Reserve huge pages. The size specified is in bytes, with
> > +        optional K, M or G suffix. The size must be a multiple
> > +        of the page size.
> > +
> > +    -p, --pagesize
> > +        Choose page size to use. If not specified, the default
> > +        system page size will be used.
> > +
> > +    -m, --mount
> > +        Mount the system huge page directory %(mnt)s
> > +
> > +    -u, --umount
> > +        Unmount the system huge page directory %(mnt)s
> > +
> > +
> > +Examples:
> > +---------
> > +
> > +To display current huge page settings:
> > +    %(argv0)s -s
> > +
> > +To a complete setup of with 2 Gigabyte of 1G huge pages:
> > +    %(argv0)s -p 1G --setup 2G
> > +
> > +Equivalent to:
> > +    %(argv0)s -p 1G -c -u -r 2G -m
> > +
> > +To clear existing huge page settings and umount %(mnt)s
> > +    %(argv0)s -c -u
> > +
> > +    """ % locals())
> > +
> > +
> > +def fmt_memsize(sz):
> > +    '''Format memory size in conventional format'''
> > +    sz_kb = int(sz)
> > +    if sz_kb >= 1024 * 1024:
> > +        return '{}Gb'.format(sz_kb / (1024 * 1024))
> > +    elif sz_kb >= 1024:
> > +        return '{}Mb'.format(sz_kb / 1024)
> > +    else:
> > +        return '{}Kb'.format(sz_kb)
> 
> I've lost count how many times i've had to reimplement this code, but there
> is an easier way :) Off the top of my head,
> 
> idx = log2(sz)
> # every 10th power of 2
> return '{}{}b'.format(sz, ' kMG'[int(idx) / 10])
> 
> or something close to that.
> 

Another minor nit, since these are memory sizes, not bandwidth rates, it's
bytes not bits, so the "b" should be "B" in all the prints, whatever way
it's calculated.