[RFC 2/3] doc: add programmer's guide for flow rule compiler

[Stephen Hemminger]

Add a chapter to the programmer's guide describing the new
rte_flow_compile library: API summary, BNF grammar, field
qualifier semantics (is/spec/last/mask/prefix), diagnostic
format, and the table-driven extension model for adding
items and actions.

Documents the limitations of the initial implementation
(item/action coverage, missing RSS and RAW handling) and
the textual MAC and IPv6 forms accepted.

Signed-off-by: Stephen Hemminger <stephen at networkplumber.org>
---
 MAINTAINERS                                |   6 +
 doc/guides/prog_guide/flow_compile_lib.rst | 272 +++++++++++++++++++++
 doc/guides/prog_guide/index.rst            |   1 +
 doc/guides/rel_notes/release_26_07.rst     |   6 +
 4 files changed, 285 insertions(+)
 create mode 100644 doc/guides/prog_guide/flow_compile_lib.rst

diff --git a/MAINTAINERS b/MAINTAINERS
index 0f5539f851..4923e126df 100644
--- a/MAINTAINERS
+++ b/MAINTAINERS
@@ -448,6 +448,12 @@ F: app/test-pmd/cmdline_flow.c
 F: doc/guides/prog_guide/ethdev/flow_offload.rst
 F: lib/ethdev/rte_flow*
 
+Flow Compiler API
+M: Stephen Hemminger <stephen at networkplumber.org>
+T: git://dpdk.org/next/dpdk-next-net
+F: lib/flow_compile/
+F: doc/guides/prog_guide/flow_compile_lib.rst
+
 Traffic Management API
 M: Cristian Dumitrescu <cristian.dumitrescu at intel.com>
 T: git://dpdk.org/next/dpdk-next-net
diff --git a/doc/guides/prog_guide/flow_compile_lib.rst b/doc/guides/prog_guide/flow_compile_lib.rst
new file mode 100644
index 0000000000..8af374b33b
--- /dev/null
+++ b/doc/guides/prog_guide/flow_compile_lib.rst
@@ -0,0 +1,272 @@
+..  SPDX-License-Identifier: BSD-3-Clause
+    Copyright (c) 2026 Stephen Hemminger <stephen at networkplumber.org>
+
+Flow Rule Compiler
+==================
+
+The flow rule compiler (``rte_flow_compile``) turns a textual
+description of an ``rte_flow`` rule into the
+``struct rte_flow_attr`` / ``struct rte_flow_item`` /
+``struct rte_flow_action`` arrays accepted by ``rte_flow_create()``.
+
+It is modelled on ``pcap_compile()`` from libpcap: a single string in,
+an opaque compiled object out, with human readable diagnostics
+written to a caller supplied buffer.
+
+The compiler depends only on the EAL and the existing
+``rte_ethdev`` (``rte_flow.h``) library.  In particular it does not
+pull in ``rte_cmdline``, so it is suitable for use from libraries,
+control planes and unit tests.
+
+
+Example
+-------
+
+.. code-block:: c
+
+   char errbuf[RTE_FLOW_COMPILE_ERRBUF_SIZE];
+   const char *src =
+       "ingress group 0 priority 1 "
+       "pattern eth / ipv4 src is 10.0.0.1 / udp dst is 4789 / end "
+       "actions queue index 3 / count / end";
+
+   struct rte_flow_compile *fc = rte_flow_compile(src, errbuf);
+   if (fc == NULL) {
+           fprintf(stderr, "%s\n", errbuf);
+           return -1;
+   }
+
+   struct rte_flow_error err;
+   struct rte_flow *f = rte_flow_compile_create(port_id, fc, &err);
+
+   /* fc may be reused on multiple ports or freed now. */
+   rte_flow_compile_free(fc);
+
+
+API summary
+-----------
+
+.. code-block:: c
+
+   struct rte_flow_compile *
+   rte_flow_compile(const char *str,
+                    char errbuf[RTE_FLOW_COMPILE_ERRBUF_SIZE]);
+
+   void
+   rte_flow_compile_free(struct rte_flow_compile *fc);
+
+   const struct rte_flow_attr   *rte_flow_compile_attr(...);
+   const struct rte_flow_item   *rte_flow_compile_pattern(..., unsigned int *n);
+   const struct rte_flow_action *rte_flow_compile_actions(..., unsigned int *n);
+
+   int             rte_flow_compile_validate(uint16_t port_id, ..., struct rte_flow_error *);
+   struct rte_flow *rte_flow_compile_create  (uint16_t port_id, ..., struct rte_flow_error *);
+
+The compiled object owns every buffer it returns: attributes,
+patterns, actions and all underlying spec/mask/last/conf payloads.
+Pointers are valid until ``rte_flow_compile_free()`` is called.
+A single compiled rule may be installed on many ports and validated
+or created concurrently from multiple threads; the parser itself
+holds no static mutable state.
+
+
+Grammar
+-------
+
+The grammar is pure ASCII; ``#`` starts an end-of-line comment.
+Whitespace is insignificant.
+
+.. code-block:: bnf
+
+   rule         ::= attribute* "pattern" item-list "actions" action-list
+   attribute    ::= "ingress" | "egress" | "transfer"
+                  | "group"    UINT
+                  | "priority" UINT
+   item-list    ::= ( item "/" )* "end"
+   item         ::= IDENT field-spec*
+   field-spec   ::= IDENT qualifier value
+   qualifier    ::= "is" | "spec" | "last" | "mask" | "prefix"
+   action-list  ::= ( action "/" )* "end"
+   action       ::= IDENT param*
+   param        ::= IDENT value
+   value        ::= UINT | IPV4 | IPV6 | MAC | HEXSTR | STRING
+
+Both lists may be empty; ``pattern end`` is a wildcard match and is
+useful as a catch-all rule.  An empty action list is accepted by the
+compiler but typically rejected by the underlying PMD.
+
+Lexical tokens:
+
+.. code-block:: bnf
+
+   IDENT        ::= [A-Za-z_][A-Za-z0-9_]*
+   UINT         ::= [0-9]+ | "0x" [0-9A-Fa-f]+         ; up to 16 hex digits
+   IPV4         ::= UINT "." UINT "." UINT "." UINT    ; each 0..255
+   IPV6         ::= RFC 4291 / 5952 textual form (no embedded IPv4)
+   MAC          ::= XX ":" XX ":" XX ":" XX ":" XX ":" XX
+   HEXSTR       ::= "0x" [0-9A-Fa-f]{2*N}              ; > 16 hex digits
+   STRING       ::= '"' character* '"'
+
+The grammar follows ``testpmd`` closely so that flow rules already
+familiar to users carry over; the lexer and parser are independent
+implementations and do not depend on testpmd, ``rte_cmdline`` or
+``cmdline_parse_*``.
+
+
+Field qualifier semantics
+-------------------------
+
+For each parsed ``field qualifier value`` triple the compiler writes
+into one or more of the spec/mask/last buffers.  Semantics match
+``testpmd``:
+
+.. list-table::
+   :header-rows: 1
+   :widths: 10 30 30 20
+
+   * - Qualifier
+     - spec
+     - mask
+     - last
+   * - ``is``
+     - value
+     - all-ones over the field
+     - --
+   * - ``spec``
+     - value
+     - --
+     - --
+   * - ``mask``
+     - --
+     - value
+     - --
+   * - ``last``
+     - --
+     - --
+     - value
+   * - ``prefix``
+     - --
+     - high N bits set (CIDR style); IPv4/IPv6 only
+     - --
+
+Last write wins.  ``ipv4 src spec 10.0.0.0 src prefix 16`` therefore
+matches the entire ``10.0.0.0/16`` range with mask ``255.255.0.0``;
+``src is 10.0.0.0`` would have set the mask to all-ones, which is
+exact match.
+
+
+Diagnostics
+-----------
+
+Errors are reported as ``LINE:COL: message`` in the caller-supplied
+``errbuf`` of at least ``RTE_FLOW_COMPILE_ERRBUF_SIZE`` (256) bytes.
+The first error wins; subsequent errors are suppressed so that the
+user sees the original cause rather than a cascade.
+
+On failure ``rte_errno`` is set to ``EINVAL`` for parse errors and
+``ENOMEM`` for allocation failures.
+
+
+Extending the compiler
+----------------------
+
+The parser is entirely table driven.  Adding a new flow item type
+requires no parser changes:
+
+#. In ``flow_compile_tables.c``, define a static
+   ``struct field_desc`` array describing the parsable fields of the
+   item's spec struct.
+#. Add an ``ITEM(...)`` entry to ``flow_items[]``.
+
+Each ``field_desc`` lists the field's offset, byte width and a
+``field_kind`` (``FK_U32``, ``FK_BE16``, ``FK_MAC``, ``FK_IPV4``,
+``FK_IPV6``, ``FK_BYTES``, ...).  Default setters honor every kind
+and produce the correct byte order automatically.
+
+For fields whose layout cannot be expressed as a plain byte range
+(C bitfields, indirect arrays, RSS keys, ...) populate the ``set``
+function pointer.  The custom setter receives the destination
+buffer, an optional mask buffer (non-NULL when the user wrote
+``is``) and the parsed value token.
+
+Adding a new action type follows the same pattern with
+``flow_actions[]`` and ``ACTION(...)``.
+
+
+Source layout
+-------------
+
+The library sits in ``lib/flow_compile`` and is split for clarity:
+
+================================  ==================================
+File                              Contents
+================================  ==================================
+``rte_flow_compile.h``            Public API.
+``flow_compile_priv.h``           Internal types: tokens, descriptors,
+                                  parser state.
+``flow_compile_lex.c``            Hand-rolled lexer with
+                                  source-position tracking for
+                                  diagnostics.
+``flow_compile_parse.c``          Recursive-descent parser plus the
+                                  default field setters used by the
+                                  table-driven body parser.
+``flow_compile_tables.c``         Per-item and per-action descriptor
+                                  tables.  All extension work
+                                  happens here.
+``rte_flow_compile_api.c``        Public entry points: compile,
+                                  free, accessors, validate, create.
+================================  ==================================
+
+
+Implementation notes
+--------------------
+
+Locale independence
+   Every character classification uses inline ASCII-only predicates
+   rather than ``<ctype.h>``; hex parsing uses an inline nibble
+   helper rather than ``strtoul()``.  The grammar is pure ASCII, so
+   the active locale cannot affect parsing.
+
+Endianness
+   All multibyte writes go through ``rte_cpu_to_be_{16,32,64}`` or
+   raw byte copies from already network-order tokens
+   (``TK_IPV4``, ``TK_MAC``, ``TK_IPV6``).
+
+Alignment
+   Spec and mask buffers may contain unaligned multibyte fields
+   inside packed-ish header structs.  All writes go through
+   ``memcpy`` to handle this portably.
+
+Memory
+   All allocations go through ``rte_zmalloc`` and ``rte_free``.  Each
+   spec, mask, last and conf payload is its own allocation; the
+   pattern and action arrays are separate ``rte_calloc`` allocations
+   that grow by doubling.  ``rte_flow_compile_free()`` walks the
+   pattern and action arrays and frees every non-NULL slot before
+   freeing the arrays themselves, so a partially compiled rule on
+   a parse-error path is cleaned up uniformly.
+
+Reentrancy
+   The parser holds no static mutable state.  Multiple threads may
+   compile rules in parallel and a single compiled rule may be
+   installed concurrently on multiple ports.
+
+
+Limitations
+-----------
+
+The initial implementation covers the most common items
+(``eth``, ``vlan``, ``ipv4``, ``ipv6``, ``tcp``, ``udp``, ``vxlan``,
+``port_id``, ``port_representor``, ``represented_port``) and actions
+(``drop``, ``passthru``, ``queue``, ``mark``, ``jump``, ``count``,
+``port_id``, ``port_representor``, ``represented_port``,
+``of_pop_vlan``, ``vxlan_decap``).  Adding more is purely a matter
+of extending the descriptor tables.
+
+Items and actions whose conf has a variable-length payload
+(``RSS``, ``RAW``, the various ``RAW_ENCAP``/``RAW_DECAP`` actions)
+are not yet wired up; they require custom setters via the
+``field_desc.set`` hook.
+
+The IPv6 tokeniser does not accept the embedded-IPv4 dotted-quad
+form (``::ffff:10.0.0.1``); use the all-hex form instead.
diff --git a/doc/guides/prog_guide/index.rst b/doc/guides/prog_guide/index.rst
index e6f24945b0..3476dfecfd 100644
--- a/doc/guides/prog_guide/index.rst
+++ b/doc/guides/prog_guide/index.rst
@@ -121,6 +121,7 @@ Utility Libraries
 
     argparse_lib
     cmdline
+    flow_compile_lib
     ptr_compress_lib
     timer_lib
     rcu_lib
diff --git a/doc/guides/rel_notes/release_26_07.rst b/doc/guides/rel_notes/release_26_07.rst
index f012d47a4b..addb9ff94b 100644
--- a/doc/guides/rel_notes/release_26_07.rst
+++ b/doc/guides/rel_notes/release_26_07.rst
@@ -64,6 +64,12 @@ New Features
   * ``--auto-probing`` enables the initial bus probing, which is the current default behavior.
 
 
+* **Added library to compile flow definitions.**
+
+  New library that works like libpcap ``pcap_compile`` function to compile
+  a text string into flow rules.
+
+
 Removed Items
 -------------
 
-- 
2.53.0