[PATCH v1 00/13] graph enhancement for multi-core dispatch

[Zhirun Yan]

Currently, rte_graph supports RTC (Run-To-Completion) model within each
of a single core.
RTC is one of the typical model of packet processing. Others like
Pipeline or Hybrid are lack of support.

The patch set introduces a 'generic' model selection which is a
self-reacting scheme according to the core affinity.
The new model enables a cross-core dispatching mechanism which employs a
scheduling work-queue to dispatch streams to other worker cores which
being associated with the destination node. When core flavor of the
destination node is a default 'current', the stream can be continue
executed as normal.

Example:
3-node graph targets 3-core budget

Generic Model
RTC:
Config Graph-A: node-0->current; node-1->current; node-2->current;
Graph-A':node-0/1/2 @0, Graph-A':node-0/1/2 @1, Graph-A':node-0/1/2 @2

+ - - - - - - - - - - - - - - - - - - - - - +
'                Core #0/1/2                '
'                                           '
' +--------+     +---------+     +--------+ '
' | Node-0 | --> | Node-1  | --> | Node-2 | '
' +--------+     +---------+     +--------+ '
'                                           '
+ - - - - - - - - - - - - - - - - - - - - - +

Pipeline:
Config Graph-A: node-0->0; node-1->1; node-2->2;
Graph-A':node-0 @0, Graph-A':node-1 @1, Graph-A':node-2 @2

+ - - - - - -+     +- - - - - - +     + - - - - - -+
'  Core #0   '     '  Core #1   '     '  Core #2   '
'            '     '            '     '            '
' +--------+ '     ' +--------+ '     ' +--------+ '
' | Node-0 | ' --> ' | Node-1 | ' --> ' | Node-2 | '
' +--------+ '     ' +--------+ '     ' +--------+ '
'            '     '            '     '            '
+ - - - - - -+     +- - - - - - +     + - - - - - -+

Hybrid:
Config Graph-A: node-0->current; node-1->current; node-2->2;
Graph-A':node-0/1 @0, Graph-A':node-0/1 @1, Graph-A':node-2 @2

+ - - - - - - - - - - - - - - - +     + - - - - - -+
'            Core #0            '     '  Core #2   '
'                               '     '            '
' +--------+         +--------+ '     ' +--------+ '
' | Node-0 | ------> | Node-1 | ' --> ' | Node-2 | '
' +--------+         +--------+ '     ' +--------+ '
'                               '     '            '
+ - - - - - - - - - - - - - - - +     + - - - - - -+
                                          ^
                                          |
                                          |
+ - - - - - - - - - - - - - - - +         |
'            Core #1            '         |
'                               '         |
' +--------+         +--------+ '         |
' | Node-0 | ------> | Node-1 | ' --------+
' +--------+         +--------+ '
'                               '
+ - - - - - - - - - - - - - - - +


The patch set has been break down as below:

1. Split graph worker into common and default model part.
2. Inline graph node processing and graph circular buffer walking to make
  it reusable.
3. Add set/get APIs to choose worker model.
4. Introduce core affinity API to set the node run on specific worker core.
  (only use in new model)
5. Introduce graph affinity API to bind one graph with specific worker
  core.
6. Introduce graph clone API.
7. Introduce stream moving with scheduler work-queue in patch 8,9,10.
8. Add stats for new models.
9. Abstract default graph config process and integrate new model into
  example/l3fwd-graph. Add new parameters for model choosing.

We could run with new worker model by this:
./dpdk-l3fwd-graph -l 8,9,10,11 -n 4 -- -p 0x1 --config="(0,0,9)" -P
--model="generic"

References:
https://static.sched.com/hosted_files/dpdkuserspace22/a6/graph%20introduce%20remote%20dispatch%20for%20mult-core%20scaling.pdf

Zhirun Yan (13):
  graph: split graph worker into common and default model
  graph: move node process into inline function
  graph: add macro to walk on graph circular buffer
  graph: add get/set graph worker model APIs
  graph: introduce core affinity API
  graph: introduce graph affinity API
  graph: introduce graph clone API for other worker core
  graph: introduce stream moving cross cores
  graph: enable create and destroy graph scheduling workqueue
  graph: introduce graph walk by cross-core dispatch
  graph: enable graph generic scheduler model
  graph: add stats for corss-core dispatching
  examples/l3fwd-graph: introduce generic worker model

 examples/l3fwd-graph/main.c         | 218 +++++++++--
 lib/graph/graph.c                   | 179 +++++++++
 lib/graph/graph_debug.c             |   6 +
 lib/graph/graph_populate.c          |   1 +
 lib/graph/graph_private.h           |  44 +++
 lib/graph/graph_stats.c             |  74 +++-
 lib/graph/meson.build               |   3 +-
 lib/graph/node.c                    |   1 +
 lib/graph/rte_graph.h               |  44 +++
 lib/graph/rte_graph_model_generic.c | 179 +++++++++
 lib/graph/rte_graph_model_generic.h | 114 ++++++
 lib/graph/rte_graph_model_rtc.h     |  22 ++
 lib/graph/rte_graph_worker.h        | 516 ++------------------------
 lib/graph/rte_graph_worker_common.h | 545 ++++++++++++++++++++++++++++
 lib/graph/version.map               |   8 +
 15 files changed, 1430 insertions(+), 524 deletions(-)
 create mode 100644 lib/graph/rte_graph_model_generic.c
 create mode 100644 lib/graph/rte_graph_model_generic.h
 create mode 100644 lib/graph/rte_graph_model_rtc.h
 create mode 100644 lib/graph/rte_graph_worker_common.h

-- 
2.25.1