[dpdk-dev] [PATCH v7 00/42] Pipeline alignment with the P4 language

[David Marchand]

On Thu, Oct 1, 2020 at 12:20 PM Cristian Dumitrescu
<cristian.dumitrescu at intel.com> wrote:
>
> This patch set introduces a new pipeline type that combines the DPDK
> performance with the flexibility of the P4-16 language[1]. The new API
> can be used either by itself to code a complete software switch (SWX)
> or data plane app, or in combination with the open-source P4 compiler
> P4C [2], potentially acting as a P4C back-end, thus allowing the P4
> programs to be translated to the DPDK API and run on multi-core CPUs.
>
> Main new features:
>
> * Nothing is hard-wired, everything is dynamically defined: The packet
>   headers (i.e. protocols), the packet meta-data, the actions, the
>   tables and the pipeline itself are dynamically defined instead of
>   having to be selected from a pre-defined set.
>
> * Instructions: The actions and the life of the packet through the
>   pipeline are defined with instructions that manipulate the pipeline
>   objects mentioned above. The pipeline is the main function of the
>   packet program, with actions as subroutines triggered by the tables.
>
> * Call external plugins: Extern objects and functions can be defined
>   to call functionality that cannot be efficiently implemented with
>   the existing pipeline-oriented instruction set, such as: special
>   error detecting/correcting codes, crypto, meters, stats arrays,
>   heuristics, etc.
>
> * Better control plane interaction: Transaction-oriented table update
>   mechanism that supports multi-table atomic updates. Multiple tables
>   can be updated in a single step with only the before and after table
>   sets visible to the packets. Alignment with P4Runtime [3].
>
> * Performance: Multiple packets are in-flight within the pipeline at
>   any moment. Each packet is owned by a different time-sharing thread
>   in run-to-completion, with the thread pausing before memory access
>   operations such as packet I/O and table lookup to allow the memory
>   prefetch to complete. The instructions are verified and translated
>   at initialization time with no run-time impact. The instructions are
>   also optimized to detect and "fuse" frequently used patterns into
>   vector-like instructions transparently to the user.
>
> API deprecation and maturing roadmap:
> * The existing pipeline stable API (rte_pipeline.h) to be deprecated
>   prior to and removed as part of the DPDK 21.11 LTS release.
> * The new SWX pipeline experimental API (rte_swx_pipeline.h) to mature
>   and become stable as part of the same DPDK 21.11 LTS release.
>
> V7 changes:
> * Set copyright year to 2020 for all new files.
> * Fixed two Doxygen warnings.
>
> V6 changes:
> * Fixed issues in the example app Makefile.
> * Used rte_eth_link_speed_to_str() in the example app.
> * Added release notes update.
>
> V5 changes:
> * Upper case abberviations in some commit titles.
> * Added new example app in the MAINTAINERS file.
> * Absolutely no code changes.
>
> V4 changes:
> * Spell check fixes.
>
> V3 changes:
> * Removed the library Makefile support to align with the latest DPDK.
>
> V2 changes:
> * Updated the title and commit messages to reflect the introduction of
>   the new SWX pipeline type.
> * Added the API deprecation and maturing roadmap to the cover letter.
> * Added support for building the SWX pipeline based on specification
>   file with syntax aligned to the P4 language. The spec file may be
>   generated by the P4C compiler in the future (see patch 32). Reworked
>   the examples accordingly (see patches 39, 40 and 41).
> * Added support for the SWX sink port (used for packet drop or log)
>   when PCAP library is disabled from the build.
> * Added checks to the application CLI commands to prevent execution
>   when dependencies of the current command have previously failed (see
>   patch 38).
> * Fixed build warning for 32-bit targets due to the printing of 64-bit
>   statistics counters (see patch 38).
>
> [1] P4-16 specification: https://p4.org/p4-spec/docs/P4-16-v1.2.1.pdf
> [2] P4-16 compiler: https://github.com/p4lang/p4c
> [3] P4Runtime specification:
>     https://p4.org/p4runtime/spec/v1.2.0/P4Runtime-Spec.pdf
>
> Cristian Dumitrescu (41):
>   pipeline: add new SWX pipeline type
>   pipeline: add SWX pipeline input port
>   pipeline: add SWX pipeline output port
>   pipeline: add SWX headers and meta-data
>   pipeline: add SWX extern objects and funcs
>   pipeline: add SWX pipeline action
>   pipeline: add SWX pipeline tables
>   pipeline: add SWX pipeline instructions
>   pipeline: add SWX Rx and extract instructions
>   pipeline: add SWX Tx and emit instructions
>   pipeline: add header validate and invalidate SWX instructions
>   pipeline: add SWX move instruction
>   pipeline: add SWX DMA instruction
>   pipeline: introduce SWX add instruction
>   pipeline: introduce SWX subtract instruction
>   pipeline: introduce SWX ckadd instruction
>   pipeline: introduce SWX cksub instruction
>   pipeline: introduce SWX and instruction
>   pipeline: introduce SWX or instruction
>   pipeline: introduce SWX XOR instruction
>   pipeline: introduce SWX SHL instruction
>   pipeline: introduce SWX SHR instruction
>   pipeline: introduce SWX table instruction
>   pipeline: introduce SWX extern instruction
>   pipeline: introduce SWX jump and return instructions
>   pipeline: add SWX instruction description
>   pipeline: add SWX instruction verifier
>   pipeline: add SWX instruction optimizer
>   pipeline: add SWX pipeline query API
>   pipeline: add SWX pipeline flush
>   pipeline: add SWX table update high level API
>   pipeline: add SWX pipeline specification file
>   port: add ethernet device SWX port
>   port: add source and sink SWX ports
>   table: add exact match SWX table
>   examples/pipeline: add new example application
>   examples/pipeline: add message passing mechanism
>   examples/pipeline: add configuration commands
>   examples/pipeline: add l2fwd example
>   examples/pipeline: add l2fwd with MAC swap example
>   examples/pipeline: add VXLAN encapsulation example
>   doc: add new SWX pipeline type to release notes
>

- Fixed pipeline .map file
- Annotated/sorted the experimental symbols in .map files touched by
this series,
- Updated doxygen index to include the new headers,
- Switched the vxlan python script (coming with the pipeline example)
to python3 + fixed some pylint trivial warnings,

Series applied.


-- 
David Marchand