[dpdk-dev] [RFC] Control packet event adapter and FIFO library

[Mattias Rönnblom]

On 2021-09-01 07:41, Kaladi, Ashok K wrote:
> Dear dpdk-dev team,
> 
> We would like to propose the following RFC for your review.
> 
> A user space application may need access to the packets handled by eventdev
> based DPDK application. This application doesn't use mbuf or eventdev based
> DPDK APIs. Presently this is not possible without passing packets through
> DPDK KNI.
> 

How about using memif and the eventdev RX adapter to solve this problem?

> This RFC introduces control packet event adapter (CP adapter) and FIFO library
> which can be used by user space applications to access the packets
> handled by eventdev based DPDK application.  Linux shared memory is used to
> keep mempool (mbufs), packets and FIFOs. This allows copy-free access of
> packets to both the applications. The Rx and Tx FIFOs are used to exchange
> packets between the applications.
> 

On a control-type interface zero-copy or not shouldn't really matter, 
and also seem likely to result in higher coupling and a more error-prone 
interface between the DPDK app and the other app.

>                                      '------------------'
>                                      |     mempool      |
> '-------------'    '-----------'    |..................|    '----------'    '-----------'
> |             |    |           |    |  <--- Tx FIFOs   |    |          |    |           |
> | User space  |    |           |    |  <--- Alloc FIFO |    | Control  |    |   DPDK    |
> |    App      <---->  FIFO Lib <---->  ---> Rx FIFOs   <----> Packet   <----> Eventdev  |
> |             |    |           |    |  ---> Free FIFO  |    | Adapter  |    |   App     |
> |             |    |           |    |                  |    |          |    |           |
> '-------------'    '-----------'    |                  |    '----------'    '-----------'
>                                      '------------------'
> 
> CP adapter does the mbuf allocation and free for user space application. Mbufs
> allocated for user space application are put into alloc queue by CP adapter.
> The used mbufs are put into free queue by user space application.
> The FIFO lib translates packets between rte_mbuf format and a simplified
> format which will be used by the user application. FIFO lib doesn't use any other
> DPDK features than rte_mbuf structure. The simplified packet format contains only
> essential parameters like address of the payload, packet length, etc.
> 
> - user space app send path:  When user space application wants to send
>    a packet, it takes an mbufs from Alloc FIFO, writes packet to it and puts it
>    into Rx queue. CP adapter which polls this queue gets the packet and enqueues
>    to eventdev. Then the mbuf is freed by CP adapter.
> 
> - use space app receive path: CP adapter puts the packets to Tx FIFO,
>    user space application polls the FIFO and gets the packet. After consuming
>    the packet it puts the mbuf to be freed to free FIFO. CP  adapter regularly
>    reads the free FIFO and frees the mbufs in it.
> 
> The CP adapter can service multiple devices and queues. Each queue is
> configured with a servicing weight to control the relative frequency with which
> the adapter polls the queue, and the event fields to use when constructing
> packet events.
> 
> API Summary
> 
> FIFO APIs:
> -  FIFO device open/close.
> -  API to allocate PDU buffer from alloc queue.
> -  API to send a burst of packets to an Rx FIFO identified by FIFO id and device id.
> -  API to receive a burst of packets from Tx FIFO identified by FIFO id and device id.
> -  API to put the used buffer to Free queue. Use application will call this after
>     using the payload.
> 
> Control Packet Adapter APIs:
> -  Initialization API.
> -  Device addition and removal APIs
> -  Device configuration APIs.
> -  FIFO add and remove APIs.
> -  Event enqueue and dequeue APIs.
> 
> 
> We look forward for your feedback on this proposal. Once we have initial feedback,
> patches will be submitted for review.
> 
> 

One thing to consider is if this FIFO service should provide guaranteed 
and/or in-order delivery. That in turn probably depends on if you by 
"control" mean slow-path traffic (ARP etc), or more RPC/IPC type control 
signaling (e.g. an "add this route" or "retrieve these stats" message).

> Thanks & Regards
> Ashok Kaladi
>