[dpdk-dev] [PATCH v8 1/6] ethdev: introduce shared Rx queue

[Andrew Rybchenko]

On 10/18/21 3:59 PM, Xueming Li wrote:
> In current DPDK framework, each Rx queue is pre-loaded with mbufs to
> save incoming packets. For some PMDs, when number of representors scale
> out in a switch domain, the memory consumption became significant.
> Polling all ports also leads to high cache miss, high latency and low
> throughput.
> 
> This patch introduce shared Rx queue. Ports in same Rx domain and
> switch domain could share Rx queue set by specifying non-zero sharing
> group in Rx queue configuration.
> 
> Shared Rx queue is identified by share_rxq field of Rx queue
> configuration. Port A RxQ X can share RxQ with Port B RxQ Y by using
> same shared Rx queue ID.
> 
> No special API is defined to receive packets from shared Rx queue.
> Polling any member port of a shared Rx queue receives packets of that
> queue for all member ports, port_id is identified by mbuf->port. PMD is
> responsible to resolve shared Rx queue from device and queue data.
> 
> Shared Rx queue must be polled in same thread or core, polling a queue
> ID of any member port is essentially same.
> 
> Multiple share groups are supported. Device should support mixed
> configuration by allowing multiple share groups and non-shared Rx queue
> on one port.
> 
> Example grouping and polling model to reflect service priority:
>  Group1, 2 shared Rx queues per port: PF, rep0, rep1
>  Group2, 1 shared Rx queue per port: rep2, rep3, ... rep127
>  Core0: poll PF queue0
>  Core1: poll PF queue1
>  Core2: poll rep2 queue0
> 
> PMD advertise shared Rx queue capability via RTE_ETH_DEV_CAPA_RXQ_SHARE.
> 
> PMD is responsible for shared Rx queue consistency checks to avoid
> member port's configuration contradict to each other.
> 
> Signed-off-by: Xueming Li <xuemingl at nvidia.com>

with few nits below:

Reviewed-by: Andrew Rybchenko <andrew.rybchenko at oktetlabs.ru>

[snip]

> diff --git a/doc/guides/prog_guide/switch_representation.rst b/doc/guides/prog_guide/switch_representation.rst
> index ff6aa91c806..fe89a7f5c33 100644
> --- a/doc/guides/prog_guide/switch_representation.rst
> +++ b/doc/guides/prog_guide/switch_representation.rst
> @@ -123,6 +123,17 @@ thought as a software "patch panel" front-end for applications.
>  .. [1] `Ethernet switch device driver model (switchdev)
>         <https://www.kernel.org/doc/Documentation/networking/switchdev.txt>`_
>  
> +- For some PMDs, memory usage of representors is huge when number of
> +  representor grows, mbufs are allocated for each descriptor of Rx queue.
> +  Polling large number of ports brings more CPU load, cache miss and
> +  latency. Shared Rx queue can be used to share Rx queue between PF and
> +  representors among same Rx domain. ``RTE_ETH_DEV_CAPA_RXQ_SHARE`` in
> +  device info is used to indicate the capability. Setting non-zero share
> +  group in Rx queue configuration to enable share, share_qid is used to
> +  identifiy the shared Rx queue in group. Polling any member port can

identifiy -> identify

> +  receive packets of all member ports in the group, port ID is saved in
> +  ``mbuf.port``.
> +
>  Basic SR-IOV
>  ------------
>  
> diff --git a/doc/guides/rel_notes/release_21_11.rst b/doc/guides/rel_notes/release_21_11.rst
> index d5435a64aa1..2143e38ff11 100644
> --- a/doc/guides/rel_notes/release_21_11.rst
> +++ b/doc/guides/rel_notes/release_21_11.rst
> @@ -75,6 +75,12 @@ New Features
>      operations.
>    * Added multi-process support.
>  
> +* **Added ethdev shared Rx queue support.**
> +
> +  * Added new device capability flag and rx domain field to switch info.

rx -> Rx

> +  * Added share group and share queue ID to Rx queue configuration.
> +  * Added testpmd support and dedicate forwarding engine.
> +
>  * **Added new RSS offload types for IPv4/L4 checksum in RSS flow.**
>  
>    Added macros ETH_RSS_IPV4_CHKSUM and ETH_RSS_L4_CHKSUM, now IPv4 and

[snip]