[dpdk-dev] [PATCH v14 0/6] Expanded statistics reporting
remy.horton at intel.com
Thu Mar 30 23:00:56 CEST 2017
This patchset consists of three libraries: A Metrics library for
distributing device information, a library that calculates bit-rate
statistics, and a library that calculates latency statistics. The
latter two libraries make use of the first library.
The Metrics library implements a mechanism by which producers can
publish numeric information for later querying by consumers.
In practice producers will typically be other libraries or
primary processes, whereas consumers will typically be applications.
Metrics themselves are statistics that are not generated by PMDs.
Metric information is populated using a push model, where producers
update the values contained within the metric library by calling an
update function on the relevant metrics. Consumers receive metric
information by querying the central metric data, which is held
in shared memory so it can also be accessed by secondary processes.
For each metric, a separate value is maintained for each port id,
and when publishing metric values the producers need to specify
which port is being updated. In addition there is a special id
RTE_METRICS_GLOBAL that is intended for global statistics that are
not associated with any individual device. Since the metrics
library is self-contained, the only restriction on port numbers is
that they are less than RTE_MAX_ETHPORTS - there is no requirement
for the ports to actually exist.
Metrics must first be registered, which is the way producers declare
the names of the metrics they will be publishing. Registration can
either be done individually, or as a group as a metric set. The
intention is for all metrics in a set to be updated in one go,
although it is also possible for metrics within a set to be updated
individually. It is up to the producers to update metrics as required.
Bit-rate statistics library
The bit-rate library calculates the mean, the exponentially-weighted
moving average, and peak bit-rates for each active port (i.e. network
device). These statistics are then reported via the metrics library
using the following names: mean_bits_in, mean_bits_out, ewma_bits_in,
ewma_bits_out, peak_bits_in, and peak_bits_out. The sampling window
used for calculation is decided by the application requiring the statistics.
Latency statistics library
The latency statistics library calculates the port-to-port latency of
packet processing by a DPDK application, reporting the minimum, average,
and maximum nano-seconds that packet processing takes, as well as the
jitter in processing delay. These statistics are then reported via the
metrics library using the following names: min_latency_ns, avg_latency_ns,
mac_latency_ns, and jitter_ns.
This is dependent on http://dpdk.org/dev/patchwork/patch/21607/ for the
timestamp MBUF field.
For more details on the metrics library and the Bit-rate and Latency
components see the Programmer's Guide updates in the patchset.
Series-acked-by: Reshma Pattan <reshma.pattan at intel.com>
* Rebased to 94c0776b1bad
* Added library optimisation flags
* Added missing ifdef guards
* Made use of ARRAY_SIZE() macro
* Corrected printf message
* Rebased to latest master (b88161be4227)
* Updated copyright years to 2017
* Some release notes were in wrong file (17.02 rather than 17.05)
* Added a missing NULL check
* Doxygen & comment spelling corrections
* Authorship of some patches changed to reflect main contributer
* EWMA calculation for inbound and outbound bitrates were subtly different
* Use sizeof() rather than magic numbers for array sizes
* The recent feb9f680cd2c ("mk: optimize directory dependencies") required
DEPDIRS-librte_ dependency information to be added to lib/Makefile
* Rebased to latest master
* Misspellings in MAINTAINERS corrected
* Removed MBUF modifications (use Olivier's MBUF rework)
* .map references to 17.02 changed to 17.05
* Release ntoes moved to release_17_05.rst
* Bit-rate library now also gives unfiltered average
* Relocated some config-related directives.
* Removed incorrect capitalisations in API docs.
* Formatting & detail corrections in release notes.
* Moved around struct member descriptions.
* Rewritten rte_metrics.h file description.
* Rewritten description of RTE_METRICS_GLOBAL.
* Used 'producers' and 'consumers' as terms.
* Removed markup (bold text) in Doxygen tags.
* Added programming guide section.
* Updated .map files to reflect function changes in v8
* Fixed rte_malloc() of zero bytes in proc_info when no metrics exist
* Fixed rte_metrics_init not being called explicitly in testpmd
* Release notes correction
* Updated copyright years
* rte_metric_init() takes socket id & must be explicitly called
* rte_metrics_reg_metric renamed to rte_metrics_reg_name()
* rte_metrics_update_metric() renamed to rte_metrics_update_value()
* Doxygen updates
* Changed malloc()/free() to rte_malloc()/rte_free()
* Removed redundant memset()
* rte_stats_bitrates_s renamed to rte_stats_bitrates_s
* Split mbuf change to own patch for visibility
* CYCLES_PER_NS now a static inline function
* latency: "hidden" pthread creation now has polling API instead.
* Struct declarations and variable definitions cleaned up
* Double initialization of the latency library now returns -EEXIST
* MAINTAINERS entry for layenctstats in correct section
* RTE_METRICS_NONPORT renamed to RTE_METRICS_GLOBAL
* Multiple changes to rte_metrics.h doxygen documentation
* Split apart latency patch into lib, test-pmd, & proc_info parts
* Reordered patches by functionality
* Insufficent capacity return value changed from -ERANGE to actual size
* Cache alignment in bitrate library
* Tightened up const usage to avoid STATIC_CONST_CHAR_ARRAY warning
* Reshma reinstated as author for (now split) latency patch
* Rebase to master
* Metrics display now has "Non port specific" rather than "port -1"
* Fixed sign issue in EWMA delta calculation
* Rebased to master
* Updated Shared Library Versions in release notes
* Merged in Reshma's latencystats library
* References to 16.11 changed to 17.02
* Fetching of non-port values was broken
* Added sanity checks to value fetching
* rte_stat_value renamed to rte_metric_value
* Corrected doxygen descriptions
* Added MAINTAINERS entries
* Added #ifdef directives to bitrate code in test-pmd
* Marked rte_stats_bitrate_s as internal
* Minor integer roundoff correction
* Coding style corrections
* Removed spurious object allocation
* Changes to rte_metrics.[ch] moved from Patch 2/3 to 1/3.
* Reintroduced non-port values (RTE_METRICS_NONPORT)
* Added spinlocks to metric library
* Removed spurious test registration/update
* Added release notes entries
* Uses a new metrics library rather than being part of ethdev
Remy Horton (3):
lib: add information metrics library
lib: add bitrate statistics library
app/test-pmd: add bitrate statistics calculation
Reshma Pattan (3):
app/proc_info: add metrics displaying
lib: added new library for latency stats
app/test-pmd: add latency statistics calculation
MAINTAINERS | 12 +
app/proc_info/main.c | 79 ++++-
app/test-pmd/parameters.c | 22 +-
app/test-pmd/testpmd.c | 78 ++++-
app/test-pmd/testpmd.h | 8 +-
config/common_base | 15 +
doc/api/doxy-api-index.md | 5 +-
doc/api/doxy-api.conf | 5 +-
doc/guides/prog_guide/index.rst | 3 +-
doc/guides/prog_guide/metrics_lib.rst | 299 +++++++++++++++++
doc/guides/rel_notes/release_17_05.rst | 21 ++
lib/Makefile | 6 +
lib/librte_bitratestats/Makefile | 53 +++
lib/librte_bitratestats/rte_bitrate.c | 146 +++++++++
lib/librte_bitratestats/rte_bitrate.h | 84 +++++
.../rte_bitratestats_version.map | 9 +
lib/librte_latencystats/Makefile | 56 ++++
lib/librte_latencystats/rte_latencystats.c | 361 +++++++++++++++++++++
lib/librte_latencystats/rte_latencystats.h | 154 +++++++++
.../rte_latencystats_version.map | 11 +
lib/librte_metrics/Makefile | 51 +++
lib/librte_metrics/rte_metrics.c | 302 +++++++++++++++++
lib/librte_metrics/rte_metrics.h | 240 ++++++++++++++
lib/librte_metrics/rte_metrics_version.map | 13 +
mk/rte.app.mk | 3 +
25 files changed, 2028 insertions(+), 8 deletions(-)
create mode 100644 doc/guides/prog_guide/metrics_lib.rst
create mode 100644 lib/librte_bitratestats/Makefile
create mode 100644 lib/librte_bitratestats/rte_bitrate.c
create mode 100644 lib/librte_bitratestats/rte_bitrate.h
create mode 100644 lib/librte_bitratestats/rte_bitratestats_version.map
create mode 100644 lib/librte_latencystats/Makefile
create mode 100644 lib/librte_latencystats/rte_latencystats.c
create mode 100644 lib/librte_latencystats/rte_latencystats.h
create mode 100644 lib/librte_latencystats/rte_latencystats_version.map
create mode 100644 lib/librte_metrics/Makefile
create mode 100644 lib/librte_metrics/rte_metrics.c
create mode 100644 lib/librte_metrics/rte_metrics.h
create mode 100644 lib/librte_metrics/rte_metrics_version.map
More information about the dev