[dpdk-dev] [PATCH v11 00/11] introduce fail-safe PMD

[Gaetan Rivet]

This PMD intercepts and manages Ethernet device removal events issued by
slave PMDs and re-initializes them transparently when brought back so that
existing applications do not need to be modified to benefit from true
hot-plugging support.

The stacked PMD approach shares many similarities with the bonding PMD but
with a different purpose. While bonding provides the ability to group
several links into a single logical device for enhanced throughput and
supports fail-over at link level, this one manages the sudden disappearance
of the underlying device; it guarantees applications face a valid device in
working order at all times.

Each fail-safe instance is configured to run atop one or several
devices, with one defined as the preferred device. Hot-plug events are
handled on all of them, and Tx is always directed to the preferred device
if present or to the next available failover device (Rx is always performed
on all devices for simplicity).

Moreover, the configured slaves (preferred or failover) do not need to be
present at initialization time and may appear later.

Slaves configuration is continuously synchronized with that of the virtual
device, which exposes their common set of capabilities to the application.
Failure to apply the current configuration state to a slave for any reason
simply reschedules its initialization.

This patchset depends on:

[dpdk-dev] [PATCH v4 0/8] fix hotplug API
http://dpdk.org/ml/archives/dev/2017-July/071340.html

v1 --> v2:

  - Wrote documentation
  - Fixed commit logs, signed-off-by
  - Added LSC event support
  - A few minor fixes

v2 --> v3:

  - Numerous bug fixes.
  - Complete sub-EAL rework to follow new bus API.
  - burst protection on sub removal.
  - more flexible sub definition.
  - flow isolated mode support.

v3 --> v4:

  - Split back commits
       net/failsafe: add fast burst functions
       net/failsafe: support device removal
    That were squashed by error during a rebase
  - Fix segfault on port plugin
  - Fix isolate mode support for MLX4 ports plugin

v4 --> v5:

  - Follow new plug / unplug API.

v5 --> v6:

  - Follow new hotplug API.
  - Improve usability of hotplug API.
  - Fix rte_dev hotplug API implementation.
  - Introduce rte_eal_devargs_rmv API as EXPERIMENTAL.
  - Use it to clean up resources on hotplug_remove.
  - Fix hotplug implementation and support un pci bus.
    The scan was not idempotent, nor clean.
    Neither were the device fields.
  - Implement plug operation for vdev bus.
    This is needed for hotplug support and to make the EAL
    independent from vdev-specific API.
  - Remove useless parameters from plug / unplug API.

This patchset is fairly big and complex. The hotplug API has been rushed and
has never been tested outside of the special case of vdev bus.

These evolutions are proposed alongside this PMD as only this PMD allows to test
this API at the moment, and without those evolutions this PMD cannot be used.

v6 --> v7

  - Split the series in three, with
    vlan filter configuration store standing alone
    Hotplug API fixes separated
    Fail-safe PMD on its own.

    These changes should ease integration.

v7 --> v8

  - Fix usage examples in documentation.
  - Use rte_eth_dev_allocated when relevant.

v8 --> v9

  - Use calloc instead of rte_realloc for command line parsing.
  - Include errno only when necessary.

v9 --> v10

  - Fix rte_flow description double-free issue.
    Thanks Raslan for finding the issue :)
  - Use RTE_ASSERT instead of assert.
    Integrate the debug log of the fail-safe to
    RTE_LOG_DEBUG macro as well, allowing dynamic log levels.
  - Remove devices post-state-sync during slave upkeep,
    to avoid a useless early probe attempt.

v10 --> v11

  - Added Glossary in documentation.
  - Fixed issues oulined in [1].
  - Removed Rx offload advertizement in feature matrix.

[1]: http://dpdk.org/ml/archives/dev/2017-July/071406.html

Gaetan Rivet (11):
  ethdev: add deferred intermediate device state
  ethdev: count devices consistently
  net/failsafe: add fail-safe PMD
  net/failsafe: add plug-in support
  net/failsafe: add flexible device definition
  net/failsafe: support flow API
  net/failsafe: support Rx offload capabilities
  net/failsafe: add fast burst functions
  net/failsafe: support device removal
  net/failsafe: support link status change event
  net/failsafe: support flow API isolation mode

 MAINTAINERS                                       |   5 +
 config/common_base                                |   5 +
 doc/guides/nics/fail_safe.rst                     | 221 ++++++
 doc/guides/nics/features/failsafe.ini             |  26 +
 doc/guides/nics/index.rst                         |   1 +
 drivers/net/Makefile                              |   2 +
 drivers/net/failsafe/Makefile                     |  60 ++
 drivers/net/failsafe/failsafe.c                   | 299 ++++++++
 drivers/net/failsafe/failsafe_args.c              | 468 ++++++++++++
 drivers/net/failsafe/failsafe_eal.c               | 118 +++
 drivers/net/failsafe/failsafe_ether.c             | 437 +++++++++++
 drivers/net/failsafe/failsafe_flow.c              | 244 ++++++
 drivers/net/failsafe/failsafe_ops.c               | 867 ++++++++++++++++++++++
 drivers/net/failsafe/failsafe_private.h           | 359 +++++++++
 drivers/net/failsafe/failsafe_rxtx.c              | 203 +++++
 drivers/net/failsafe/rte_pmd_failsafe_version.map |   4 +
 lib/librte_ether/rte_ethdev.c                     |  19 +-
 lib/librte_ether/rte_ethdev.h                     |  14 +-
 mk/rte.app.mk                                     |   1 +
 19 files changed, 3338 insertions(+), 15 deletions(-)
 create mode 100644 doc/guides/nics/fail_safe.rst
 create mode 100644 doc/guides/nics/features/failsafe.ini
 create mode 100644 drivers/net/failsafe/Makefile
 create mode 100644 drivers/net/failsafe/failsafe.c
 create mode 100644 drivers/net/failsafe/failsafe_args.c
 create mode 100644 drivers/net/failsafe/failsafe_eal.c
 create mode 100644 drivers/net/failsafe/failsafe_ether.c
 create mode 100644 drivers/net/failsafe/failsafe_flow.c
 create mode 100644 drivers/net/failsafe/failsafe_ops.c
 create mode 100644 drivers/net/failsafe/failsafe_private.h
 create mode 100644 drivers/net/failsafe/failsafe_rxtx.c
 create mode 100644 drivers/net/failsafe/rte_pmd_failsafe_version.map

-- 
2.1.4