<div dir="ltr"><div dir="ltr"><br></div><br><div class="gmail_quote gmail_quote_container"><div dir="ltr" class="gmail_attr">st 6. 5. 2026 v 5:33 odesílatel Stephen Hemminger <<a href="mailto:stephen@networkplumber.org">stephen@networkplumber.org</a>> napsal:<br></div><blockquote class="gmail_quote" style="margin:0px 0px 0px 0.8ex;border-left:1px solid rgb(204,204,204);padding-left:1ex">Background<br>
----------<br>
<br>
Multiple efforts over the past few cycles have tried to make<br>
testpmd's flow rule grammar reusable from outside testpmd.<br>
External applications that need rte_flow want a documented way<br>
to turn human-written rules into the rte_flow_attr/item/action<br>
arrays accepted by rte_flow_create().<br>
<br>
The most recent attempt is Lukas Sismis's series, currently at<br>
v12:<br>
<br>
<a href="http://patches.dpdk.org/project/dpdk/list/?series=37384" rel="noreferrer" target="_blank">http://patches.dpdk.org/project/dpdk/list/?series=37384</a> (or<br>
most recent thread on <a href="mailto:dev@dpdk.org" target="_blank">dev@dpdk.org</a>)<br>
<br>
That series factors testpmd's existing cmdline_flow.c into a<br>
library and updates testpmd to consume it. It works, but<br>
inherits two properties of cmdline_flow.c that I think are worth<br>
avoiding in a reusable library:<br>
<br>
- Coupling to librte_cmdline. Even after the v12 split into<br>
a "simple" part and a "cmdline" part, the parser is still<br>
organized around testpmd's command interpreter, and v12 has<br>
cmdline depending on ethdev to break a previous circular<br>
dependency. A library used by daemons, control planes, or<br>
unit tests should not need that.<br>
<br>
- Ad-hoc grammar. cmdline_flow.c implements parsing per-token<br>
in long dispatch logic; the grammar emerges from the code<br>
rather than being stated, and adding a new flow item<br>
requires touching the parser.<br>
<br>
This RFC explores a different shape and is posted to ask the<br>
list which one is preferred before more work goes into either.<br>
<br>
So I started a new green field library for parsing flow rules<br>
(with the help of AI assistance). It is only a few hours old,<br>
but passes tests and passes review.<br>
<br>
This series<br>
-----------<br>
<br>
lib/flow_compile -- a small new library providing the same<br>
service via a pcap_compile()-style API:<br>
<br>
char errbuf[RTE_FLOW_COMPILE_ERRBUF_SIZE];<br>
struct rte_flow_compile *fc = rte_flow_compile(rule, errbuf);<br>
if (fc == NULL)<br>
fail(errbuf); /* "line:col: message" */<br>
<br>
rte_flow_compile_create(port_id, fc, &flow_error);<br>
rte_flow_compile_free(fc);<br>
<br>
Design properties:<br>
<br>
- Hand-rolled lexer + recursive descent parser. No flex/bison.<br>
- Parser is driven entirely by descriptor tables of items and<br>
actions. Adding a new flow item is a table edit, not a<br>
parser change. A custom-setter hook on each field covers<br>
the layouts that do not fit a plain byte range (bitfields,<br>
indirect arrays).<br>
- Dependencies: rte_ethdev and rte_net only. No librte_cmdline,<br>
no flex/bison. Builds clean on Linux, FreeBSD, and Windows.<br>
- Per-allocation rte_zmalloc for spec/mask/last/conf payloads;<br>
rte_flow_compile_free() walks the pattern and action arrays.<br>
ASan/LSan run clean on the autotest.<br>
<br>
The grammar follows testpmd's syntax closely so familiar rules<br>
carry over:<br>
<br>
ingress pattern eth / ipv4 src is 10.0.0.1 / end<br>
actions queue index 3 / count / end<br>
<br>
and is documented as a formal BNF in the programmer's guide<br>
chapter (patch 2).<br>
<br>
Initial coverage: eth, vlan, ipv4, ipv6, tcp, udp, vxlan,<br>
port_id, port_representor, represented_port items; drop,<br>
passthru, queue, mark, jump, count, port_id and representor<br>
variants, of_pop_vlan, vxlan_decap actions. Variable-conf<br>
items and actions (RSS, RAW) require custom setters and are<br>
deferred to a follow-up.<br>
<br>
What this RFC is *not*<br>
----------------------<br>
<br>
Not a replacement for cmdline_flow.c in testpmd. If the shape<br>
here is acceptable, the next step is a separate series adding a<br>
"flow compile <port> <rule>" command in testpmd alongside the<br>
existing parser, so users can adopt the library incrementally<br>
without breaking scripts that depend on the current syntax.<br>
<br>
What I'd like feedback on<br>
-------------------------<br>
<br></blockquote><div><br></div><div>Hi Stephen,</div><div><br></div><div>Thank you for looking into this. I genuinely appreciate it.</div><div>I found this almost by accident, so here are some of my answers:</div><div> </div><blockquote class="gmail_quote" style="margin:0px 0px 0px 0.8ex;border-left:1px solid rgb(204,204,204);padding-left:1ex">
1. API shape. pcap_compile-style (one string -> opaque object -><br>
arrays) versus the three-call attr/pattern/actions form<br>
Sismis's v12 exposes. What does your application actually<br>
want?<br></blockquote><div><br></div><div>Generally, I am fine with pcap_compile style as per your suggestions.</div><div>The original intent was to only use traffic patterns because everything else was set by the application itself. You can see an example of the use here: <a href="https://github.com/OISF/suricata/pull/15290/changes#diff-4d5f5239a7124ed4b4d45501788bddffe803911f19c157e3c4ac7676a84d68b1R301">https://github.com/OISF/suricata/pull/15290/changes#diff-4d5f5239a7124ed4b4d45501788bddffe803911f19c157e3c4ac7676a84d68b1R301</a> </div><div><br></div><div>That is why I divided the simple API into 3 calls -- to let the application decide which part of the flow expression to parse. But a single entry point to the library is fine too, as the application could just fill it into the desired rule.</div><div><br></div><blockquote class="gmail_quote" style="margin:0px 0px 0px 0.8ex;border-left:1px solid rgb(204,204,204);padding-left:1ex">
2. Library placement. Stand-alone at lib/flow_compile/ versus<br>
addition to lib/ethdev. This series treats it as a<br>
control-path parser layered on top of ethdev rather than<br>
part of ethdev itself; v12 places its parser inside ethdev.<br></blockquote><div><br></div><div>My general take on this is that it can be so useful for developers, so it should be part of the lib/ethdev to make it more visible. As highlighted by Thomas, it heavily interacts with the rte_flow objects.</div><div>But, as it probes for the flex/bison tools and skips the library if they are unavailable, wouldn't it be a problem if it is part of lib/ethdev directly? This can be a very strong argument for a separate library, then. </div><blockquote class="gmail_quote" style="margin:0px 0px 0px 0.8ex;border-left:1px solid rgb(204,204,204);padding-left:1ex">
<br>
3. Table-driven extension model. Is "to add a new flow item,<br>
add a row to the descriptor table" the right contract?<br>
Should the tables live alongside each rte_flow_item_*<br>
definition in rte_flow.h, or in their own file as here? </blockquote><blockquote class="gmail_quote" style="margin:0px 0px 0px 0.8ex;border-left:1px solid rgb(204,204,204);padding-left:1ex">
<br>
4. Convergence. If this design is preferred, I'm happy to<br>
coordinate with Lukas to fold in the testpmd-side changes<br>
from his series.<br></blockquote><div><br></div><div>(My) original idea was to reuse one parser for both the testpmd and the library to avoid multiple implementations of essentially the same thing.</div><div><br></div><div>How would "flow compile" be useful if testpmd already has the "flow create" command? Perhaps as a library demo only? I would expect testpmd users to like the `flow create` endpoint more for its more comprehensive support.</div><div>Similarly, at this moment, since testpmd supports interactivity, I cannot see how the `_compile` API would replace functions in testpmd. But perhaps it would be part of a greater effort to migrate <span aria-invalid="grammar" class="Lm ng" style="border-style:none;background:none">to flex</span>/bison solution.</div><div>Since this is also a simple, single-rule, stateless parser, it is not fully compatible with all of TestPMD's commands (e.g., "set raw_encap"). (Ok, noticed, you highlighted this)</div><div>Perhaps it is fine, just wanted to highlight this.</div><div><br></div><div>From the user-as-an-engineer's perspective, I would be generally happy for this parser to exist. The drop-filter feature in Suricata, in my view, requires just a simple network pattern specification so looking at the supported patterns already ticks off most of the items I've had in mind, except, e.g., MPLS. I can also see this as a viable way for Suricata user-specified decap options for better applicability of RSS. </div><div><br></div><blockquote class="gmail_quote" style="margin:0px 0px 0px 0.8ex;border-left:1px solid rgb(204,204,204);padding-left:1ex">
<br>
5. API code. Is it readable enough. Lots of code here<br>
but doing lexer/parser is prime template territory<br>
and AI has a good chance here.<br>
<br>
Stephen Hemminger (3):<br>
flow_compile: introduce textual flow rule compiler<br>
doc: add programmer's guide for flow rule compiler<br>
test/flow_compile: add unit tests for flow rule compiler<br>
<br>
MAINTAINERS | 8 +<br>
app/test/meson.build | 2 +<br>
app/test/test_flow_compile.c | 230 +++<br>
doc/guides/prog_guide/flow_compile_lib.rst | 170 ++<br>
doc/guides/prog_guide/index.rst | 1 +<br>
doc/guides/rel_notes/release_26_07.rst | 5 +<br>
lib/flow_compile/flow_compile_lex.c | 488 +++++<br>
lib/flow_compile/flow_compile_parse.c | 634 ++++++<br>
lib/flow_compile/flow_compile_priv.h | 181 ++<br>
lib/flow_compile/flow_compile_tables.c | 245 +++<br>
lib/flow_compile/meson.build | 15 +<br>
lib/flow_compile/rte_flow_compile.h | 158 ++<br>
lib/flow_compile/rte_flow_compile_api.c | 132 ++<br>
lib/flow_compile/version.map | 13 +<br>
lib/meson.build | 1 +<br>
15 files changed, 2283 insertions(+)<br>
create mode 100644 app/test/test_flow_compile.c<br>
create mode 100644 doc/guides/prog_guide/flow_compile_lib.rst<br>
create mode 100644 lib/flow_compile/flow_compile_lex.c<br>
create mode 100644 lib/flow_compile/flow_compile_parse.c<br>
create mode 100644 lib/flow_compile/flow_compile_priv.h<br>
create mode 100644 lib/flow_compile/flow_compile_tables.c<br>
create mode 100644 lib/flow_compile/meson.build<br>
create mode 100644 lib/flow_compile/rte_flow_compile.h<br>
create mode 100644 lib/flow_compile/rte_flow_compile_api.c<br>
create mode 100644 lib/flow_compile/version.map<br>
<br>
-- <br>
2.43.7<br>
<br>
<br>
Stephen Hemminger (3):<br>
flow_compile: introduce textual flow rule compiler<br>
doc: add programmer's guide for flow rule compiler<br>
test/flow_compile: add unit tests for flow rule compiler<br>
<br>
MAINTAINERS | 7 +<br>
app/test/meson.build | 1 +<br>
app/test/test_flow_compile.c | 234 ++++++++<br>
doc/guides/prog_guide/flow_compile_lib.rst | 272 +++++++++<br>
doc/guides/prog_guide/index.rst | 1 +<br>
doc/guides/rel_notes/release_26_07.rst | 6 +<br>
lib/flow_compile/flow_compile_lex.c | 510 +++++++++++++++++<br>
lib/flow_compile/flow_compile_parse.c | 634 +++++++++++++++++++++<br>
lib/flow_compile/flow_compile_priv.h | 181 ++++++<br>
lib/flow_compile/flow_compile_tables.c | 245 ++++++++<br>
lib/flow_compile/meson.build | 15 +<br>
lib/flow_compile/rte_flow_compile.h | 158 +++++<br>
lib/flow_compile/rte_flow_compile_api.c | 132 +++++<br>
lib/meson.build | 1 +<br>
14 files changed, 2397 insertions(+)<br>
create mode 100644 app/test/test_flow_compile.c<br>
create mode 100644 doc/guides/prog_guide/flow_compile_lib.rst<br>
create mode 100644 lib/flow_compile/flow_compile_lex.c<br>
create mode 100644 lib/flow_compile/flow_compile_parse.c<br>
create mode 100644 lib/flow_compile/flow_compile_priv.h<br>
create mode 100644 lib/flow_compile/flow_compile_tables.c<br>
create mode 100644 lib/flow_compile/meson.build<br>
create mode 100644 lib/flow_compile/rte_flow_compile.h<br>
create mode 100644 lib/flow_compile/rte_flow_compile_api.c<br>
<br>
-- <br>
2.53.0<br>
<br>
</blockquote></div></div>