[dpdk-dev] [PATCH v13 0/6] Expanded statistics reporting

Pattan, Reshma reshma.pattan at intel.com
Thu Mar 30 15:18:52 CEST 2017



> -----Original Message-----
> From: dev [mailto:dev-bounces at dpdk.org] On Behalf Of Remy Horton
> Sent: Wednesday, March 29, 2017 7:29 PM
> To: dev at dpdk.org
> Cc: Thomas Monjalon <thomas.monjalon at 6wind.com>
> Subject: [dpdk-dev] [PATCH v13 0/6] Expanded statistics reporting
> 
> This patchset consists of three libraries: A Metrics library for distributing
> device information, a library that calculates bit-rate statistics, and a library
> that calculates latency statistics. The latter two libraries make use of the first
> library.
> 
> Metrics Library
> ---------------
> The Metrics library implements a mechanism by which producers can publish
> numeric information for later querying by consumers.
> In practice producers will typically be other libraries or primary processes,
> whereas consumers will typically be applications.
> 
> Metrics themselves are statistics that are not generated by PMDs.
> Metric information is populated using a push model, where producers
> update the values contained within the metric library by calling an update
> function on the relevant metrics. Consumers receive metric information by
> querying the central metric data, which is held in shared memory so it can
> also be accessed by secondary processes.
> 
> For each metric, a separate value is maintained for each port id, and when
> publishing metric values the producers need to specify which port is being
> updated. In addition there is a special id RTE_METRICS_GLOBAL that is
> intended for global statistics that are not associated with any individual
> device. Since the metrics library is self-contained, the only restriction on port
> numbers is that they are less than RTE_MAX_ETHPORTS - there is no
> requirement for the ports to actually exist.
> 
> Metrics must first be registered, which is the way producers declare the
> names of the metrics they will be publishing. Registration can either be done
> individually, or as a group as a metric set. The intention is for all metrics in a
> set to be updated in one go, although it is also possible for metrics within a
> set to be updated individually. It is up to the producers to update metrics as
> required.
> 
> Bit-rate statistics library
> ---------------------------
> The bit-rate library calculates the mean, the exponentially-weighted moving
> average, and peak bit-rates for each active port (i.e. network device). These
> statistics are then reported via the metrics library using the following names:
> mean_bits_in, mean_bits_out, ewma_bits_in, ewma_bits_out, peak_bits_in,
> and peak_bits_out. The sampling window used for calculation is decided by
> the application requiring the statistics.
> 
> Latency statistics library
> --------------------------
> The latency statistics library calculates the port-to-port latency of packet
> processing by a DPDK application, reporting the minimum, average, and
> maximum nano-seconds that packet processing takes, as well as the jitter in
> processing delay. These statistics are then reported via the metrics library
> using the following names: min_latency_ns, avg_latency_ns, mac_latency_ns,
> and jitter_ns.
> 
> This is dependent on http://dpdk.org/dev/patchwork/patch/21607/ for the
> timestamp MBUF field.
> 
> For more details on the metrics library and the Bit-rate and Latency
> components see the Programmer's Guide updates in the patchset.
> --
> v13 changes:
> * Rebased to latest master (b88161be4227)
> * Updated copyright years to 2017
> * Some release notes were in wrong file (17.02 rather than 17.05)
> * Added a missing NULL check
> * Doxygen & comment spelling corrections
> * Authorship of some patches changed to reflect main contributer
> * EWMA calculation for inbound and outbound bitrates were subtly
> different
> * Use sizeof() rather than magic numbers for array sizes
> * The recent feb9f680cd2c ("mk: optimize directory dependencies") required
>   DEPDIRS-librte_ dependency information to be added to lib/Makefile
> 
> v12 changes:
> * Rebased to latest master
> * Misspellings in MAINTAINERS corrected
> * Removed MBUF modifications (use Olivier's MBUF rework)
> 
> v11 changes:
> * Rebased
> * .map references to 17.02 changed to 17.05
> * Release ntoes moved to release_17_05.rst
> * Bit-rate library now also gives unfiltered average
> 
> v10 changes:
> * Rebased
> * Relocated some config-related directives.
> * Removed incorrect capitalisations in API docs.
> * Formatting & detail corrections in release notes.
> * Moved around struct member descriptions.
> * Rewritten rte_metrics.h file description.
> * Rewritten description of RTE_METRICS_GLOBAL.
> * Used 'producers' and 'consumers' as terms.
> * Removed markup (bold text) in Doxygen tags.
> * Added programming guide section.
> 
> v9 changes:
> * Updated .map files to reflect function changes in v8
> * Fixed rte_malloc() of zero bytes in proc_info when no metrics exist
> * Fixed rte_metrics_init not being called explicitly in testpmd
> 
> v8 changes:
> * Release notes correction
> * Updated copyright years
> * rte_metric_init() takes socket id & must be explicitly called
> * rte_metrics_reg_metric renamed to rte_metrics_reg_name()
> * rte_metrics_update_metric() renamed to rte_metrics_update_value()
> * Doxygen updates
> * Changed malloc()/free() to rte_malloc()/rte_free()
> * Removed redundant memset()
> * rte_stats_bitrates_s renamed to rte_stats_bitrates_s
> * Split mbuf change to own patch for visibility
> * CYCLES_PER_NS now a static inline function
> * latency: "hidden" pthread creation now has polling API instead.
> * Struct declarations and variable definitions cleaned up
> * Double initialization of the latency library now returns -EEXIST
> * MAINTAINERS entry for layenctstats in correct section
> 
> v7 changes:
> * RTE_METRICS_NONPORT renamed to RTE_METRICS_GLOBAL
> * Multiple changes to rte_metrics.h doxygen documentation
> * Split apart latency patch into lib, test-pmd, & proc_info parts
> * Reordered patches by functionality
> * Insufficent capacity return value changed from -ERANGE to actual size
> * Cache alignment in bitrate library
> * Tightened up const usage to avoid STATIC_CONST_CHAR_ARRAY warning
> * Reshma reinstated as author for (now split) latency patch
> * Rebase to master
> 
> v6 changes:
> * Metrics display now has "Non port specific" rather than "port -1"
> * Fixed sign issue in EWMA delta calculation
> * Rebased to master
> 
> v5 changes:
> * Updated Shared Library Versions in release notes
> * Merged in Reshma's latencystats library
> 
> v4 changes:
> * References to 16.11 changed to 17.02
> * Fetching of non-port values was broken
> * Added sanity checks to value fetching
> * rte_stat_value renamed to rte_metric_value
> * Corrected doxygen descriptions
> * Added MAINTAINERS entries
> * Added #ifdef directives to bitrate code in test-pmd
> 
> v3 changes:
> * Marked rte_stats_bitrate_s as internal
> * Minor integer roundoff correction
> * Coding style corrections
> * Removed spurious object allocation
> * Changes to rte_metrics.[ch] moved from Patch 2/3 to 1/3.
> * Reintroduced non-port values (RTE_METRICS_NONPORT)
> * Added spinlocks to metric library
> * Removed spurious test registration/update
> * Added release notes entries
> 
> v2 changes:
> * Uses a new metrics library rather than being part of ethdev
> 
> 
> Remy Horton (3):
>   lib: add information metrics library
>   lib: add bitrate statistics library
>   app/test-pmd: add bitrate statistics calculation
> 
> Reshma Pattan (3):
>   app/proc_info: add metrics displaying
>   lib: added new library for latency stats
>   app/test-pmd: add latency statistics calculation
> 
>  MAINTAINERS                                        |  12 +
>  app/proc_info/main.c                               |  79 ++++-
>  app/test-pmd/parameters.c                          |  20 +-
>  app/test-pmd/testpmd.c                             |  78 ++++-
>  app/test-pmd/testpmd.h                             |   6 +-
>  config/common_base                                 |  15 +
>  doc/api/doxy-api-index.md                          |   5 +-
>  doc/api/doxy-api.conf                              |   5 +-
>  doc/guides/prog_guide/index.rst                    |   3 +-
>  doc/guides/prog_guide/metrics_lib.rst              | 299 +++++++++++++++++
>  doc/guides/rel_notes/release_17_05.rst             |  21 ++
>  lib/Makefile                                       |   6 +
>  lib/librte_bitratestats/Makefile                   |  53 +++
>  lib/librte_bitratestats/rte_bitrate.c              | 145 +++++++++
>  lib/librte_bitratestats/rte_bitrate.h              |  84 +++++
>  .../rte_bitratestats_version.map                   |   9 +
>  lib/librte_latencystats/Makefile                   |  56 ++++
>  lib/librte_latencystats/rte_latencystats.c         | 361
> +++++++++++++++++++++
>  lib/librte_latencystats/rte_latencystats.h         | 154 +++++++++
>  .../rte_latencystats_version.map                   |  11 +
>  lib/librte_metrics/Makefile                        |  51 +++
>  lib/librte_metrics/rte_metrics.c                   | 302 +++++++++++++++++
>  lib/librte_metrics/rte_metrics.h                   | 240 ++++++++++++++
>  lib/librte_metrics/rte_metrics_version.map         |  13 +
>  mk/rte.app.mk                                      |   3 +
>  25 files changed, 2023 insertions(+), 8 deletions(-)  create mode 100644
> doc/guides/prog_guide/metrics_lib.rst
>  create mode 100644 lib/librte_bitratestats/Makefile  create mode 100644
> lib/librte_bitratestats/rte_bitrate.c
>  create mode 100644 lib/librte_bitratestats/rte_bitrate.h
>  create mode 100644 lib/librte_bitratestats/rte_bitratestats_version.map
>  create mode 100644 lib/librte_latencystats/Makefile  create mode 100644
> lib/librte_latencystats/rte_latencystats.c
>  create mode 100644 lib/librte_latencystats/rte_latencystats.h
>  create mode 100644 lib/librte_latencystats/rte_latencystats_version.map
>  create mode 100644 lib/librte_metrics/Makefile  create mode 100644
> lib/librte_metrics/rte_metrics.c  create mode 100644
> lib/librte_metrics/rte_metrics.h  create mode 100644
> lib/librte_metrics/rte_metrics_version.map
> 
> --
> 2.5.5

Since I have only very few minor comments on this patch set , I will ack this patch, please feel free to add my ack for next patch set.

Series Acked-by: Reshma Pattan <reshma.pattan at intel.com>




More information about the dev mailing list