[PATCH v2 1/2] member: implement NitroSketch mode

[Rong, Leyi]

Hi David,

> -----Original Message-----
> From: David Marchand <david.marchand at redhat.com>
> Sent: Tuesday, September 13, 2022 10:56 PM
> To: Rong, Leyi <leyi.rong at intel.com>
> Cc: ferruh.yigit at xilinx.com; suanmingm at nvidia.com; Wang, Yipeng1
> <yipeng1.wang at intel.com>; zaoxingliu at gmail.com; Gobriel, Sameh
> <sameh.gobriel at intel.com>; dev at dpdk.org; Richardson, Bruce
> <bruce.richardson at intel.com>; Aaron Conole <aconole at redhat.com>; Michael
> Santana <maicolgabriel at hotmail.com>; Lincoln Lavoie <lylavoie at iol.unh.edu>
> Subject: Re: [PATCH v2 1/2] member: implement NitroSketch mode
> 
> On Wed, Aug 31, 2022 at 8:47 AM Leyi Rong <leyi.rong at intel.com> wrote:
> >
> > Sketching algorithm provide high-fidelity approximate measurements and
> > appears as a promising alternative to traditional approaches such as
> > packet sampling.
> >
> > NitroSketch [1] is a software sketching framework that optimizes
> > performance, provides accuracy guarantees, and supports a variety of
> > sketches.
> >
> > This commit adds a new data structure called sketch into membership
> > library. This new data structure is an efficient way to profile the
> > traffic for heavy hitters. Also use min-heap structure to maintain the
> > top-k flow keys.
> 
> http://mails.dpdk.org/archives/test-report/2022-August/304026.html
> 
> This patch adds new symbols in the API without going through the experimental
> phase.
> What is the rationale for skipping it?
> 
Yes,  these new APIs should be marked as "experimental", will fix it.

> >
> > [1] Zaoxing Liu, Ran Ben-Basat, Gil Einziger, Yaron Kassner, Vladimir
> > Braverman, Roy Friedman, Vyas Sekar, "NitroSketch: Robust and General
> > Sketch-based Monitoring in Software Switches", in ACM SIGCOMM 2019.
> > https://dl.acm.org/doi/pdf/10.1145/3341302.3342076
> >
> > Signed-off-by: Alan Liu <zaoxingliu at gmail.com>
> > Signed-off-by: Yipeng Wang <yipeng1.wang at intel.com>
> > Signed-off-by: Leyi Rong <leyi.rong at intel.com>
> > ---
> >  lib/member/meson.build                |  38 +-
> >  lib/member/rte_member.c               |  75 ++++
> >  lib/member/rte_member.h               | 151 ++++++-
> >  lib/member/rte_member_heap.h          | 424 ++++++++++++++++++
> >  lib/member/rte_member_sketch.c        | 594 ++++++++++++++++++++++++++
> >  lib/member/rte_member_sketch.h        |  97 +++++
> >  lib/member/rte_member_sketch_avx512.c |  69 +++
> > lib/member/rte_member_sketch_avx512.h |  36 ++
> >  lib/member/rte_xxh64_avx512.h         | 117 +++++
> >  lib/member/version.map                |   3 +
> >  10 files changed, 1600 insertions(+), 4 deletions(-)  create mode
> > 100644 lib/member/rte_member_heap.h  create mode 100644
> > lib/member/rte_member_sketch.c  create mode 100644
> > lib/member/rte_member_sketch.h  create mode 100644
> > lib/member/rte_member_sketch_avx512.c
> >  create mode 100644 lib/member/rte_member_sketch_avx512.h
> >  create mode 100644 lib/member/rte_xxh64_avx512.h
> >
> > diff --git a/lib/member/meson.build b/lib/member/meson.build index
> > e06fddc240..9b3418c25c 100644
> > --- a/lib/member/meson.build
> > +++ b/lib/member/meson.build
> > @@ -7,6 +7,42 @@ if is_windows
> >      subdir_done()
> >  endif
> >
> > -sources = files('rte_member.c', 'rte_member_ht.c',
> > 'rte_member_vbf.c')
> > +sources = files('rte_member.c', 'rte_member_ht.c',
> > +'rte_member_vbf.c', 'rte_member_sketch.c')
> >  headers = files('rte_member.h')
> >  deps += ['hash']
> > +includes += include_directories('../hash', '../ring')
> > +
> > +# compile AVX512 version if:
> > +# we are building 64-bit binary AND binutils can generate proper code
> > +if dpdk_conf.has('RTE_ARCH_X86_64') and binutils_ok
> > +    # compile AVX512 version if either:
> > +    # a. we have AVX512 supported in minimum instruction set
> > +    #    baseline
> > +    # b. it's not minimum instruction set, but supported by
> > +    #    compiler
> > +    #
> > +    # in former case, just add avx512 C file to files list
> > +    # in latter case, compile c file to static lib, using correct
> > +    # compiler flags, and then have the .o file from static lib
> > +    # linked into main lib.
> > +    sketch_avx512_cpu_support = (
> > +        cc.get_define('__AVX512F__', args: machine_args) != ''
> > +    )
> > +
> > +    if sketch_avx512_cpu_support == true
> > +       cflags += ['-DCC_AVX512_SUPPORT']
> > +       if cc.has_multi_arguments('-mavx512f', '-mavx512dq', '-mavx512ifma')
> > +           cflags += ['-mavx512f', '-mavx512dq', '-mavx512ifma']
> 
> Pushing those flags in the cflags is probably wrong, as the rest of the library
> objects will be compiled with those AVX512 flags.
> If later this library code is run on a non supporting AVX512 system, it will trigger
> a runtime error.
> 
Yes, thanks for the reminder, will fix it.

> 
> Please look at how other libraries integrated AVX512.
> Thanks.
> 
> --
> David Marchand