[dpdk-dev] [PATCH v12] app/testpmd: support multi-process

[Min Hu (Connor)]

Hi, Singh, fixed in v13, thanks.

在 2021/4/21 17:56, Singh, Aman Deep 写道:
> This patch adds multi-process support for testpmd.
> 
> The test cmd example as follows:
> 
> the primary cmd:
> 
> ./dpdk-testpmd -a xxx --proc-type=auto -l 0-1 -- -i \
> 
> --rxq=4 --txq=4 --num-procs=2 --proc-id=0
> 
> the secondary cmd:
> 
> ./dpdk-testpmd -a xxx --proc-type=auto -l 2-3 -- -i \
> 
> --rxq=4 --txq=4 --num-procs=2 --proc-id=1
> 
> Signed-off-by: Min Hu (Connor) <humin29 at huawei.com 
> <mailto:humin29 at huawei.com>>
> 
> Signed-off-by: Lijun Ou <oulijun at huawei.com <mailto:oulijun at huawei.com>>
> 
> Acked-by: Xiaoyun Li <xiaoyun.li at intel.com <mailto:xiaoyun.li at intel.com>>
> 
> Acked-by: Ajit Khaparde <ajit.khaparde at broadcom.com 
> <mailto:ajit.khaparde at broadcom.com>>
> 
> Reviewed-by: Ferruh Yigit <ferruh.yigit at intel.com 
> <mailto:ferruh.yigit at intel.com>>
> 
> ---
> 
> v12:
> 
> * Updated doc info.
> 
> <….>
> 
> diff --git a/doc/guides/testpmd_app_ug/run_app.rst 
> b/doc/guides/testpmd_app_ug/run_app.rst
> 
> index d062165..7414e85 100644
> 
> --- a/doc/guides/testpmd_app_ug/run_app.rst
> 
> +++ b/doc/guides/testpmd_app_ug/run_app.rst
> 
> @@ -543,3 +543,87 @@ The command line options are:
> 
>       bit 1 - two hairpin ports paired
> 
>       bit 0 - two hairpin ports loop
> 
>       The default value is 0. Hairpin will use single port mode and 
> implicit Tx flow mode.
> 
> +
> 
> +Testpmd Multi-Process Command-line Options
> 
> +~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
> 
> +
> 
> +The following are the command-line options for testpmd multi-process 
> support:
> 
> +
> 
> +.. code-block:: console
> 
> +
> 
> +            primary process:
> 
> +            sudo ./dpdk-testpmd -a xxx --proc-type=auto -l 0-1 -- -i 
> --rxq=4 --txq=4 \
> 
> +        --num-procs=2 --proc-id=0
> 
> +
> 
> +            secondary process:
> 
> +            sudo ./dpdk-testpmd -a xxx --proc-type=auto -l 2-3 -- -i 
> --rxq=4 --txq=4 \
> 
> +        --num-procs=2 --proc-id=1
> 
> +
> 
> +The command line options are:
> 
> +
> 
> +*   ``--rxq=N``
> 
> +
> 
> +    Set the number of Rx queues per port to N. N is the sum of queues 
> used by primary
> 
> +    and secondary process. Primary process and secondary process should 
> have separate
> 
> +    queues, and each should occupy at least one queue. Where N should 
> be the multiple
> 
> // [Minor Comment]: , not required
> 
> +    of number of processes.
> 
> +
> 
> +*   ``--txq=N``
> 
> +
> 
> +    Set the number of Tx queues per port to N. N is the sum of queues 
> used by primary
> 
> +    and secondary process. Primary process and secondary process should 
> have separate
> 
> +    queues, and each should occupy at least one queue. Where N should 
> be the multiple
> 
> // [Minor Comment]: , not required
> 
> +    of number of processes.
> 
> +
> 
> +*   ``--num-procs=N``
> 
> +
> 
> +    The number of processes which will be used.
> 
> +
> 
> +*   ``--proc-id=id``
> 
> +
> 
> +    The id of the current process (id < num-procs). id should be 
> different in primary
> 
> +    process and secondary process, which starts from '0'.
> 
> +
> 
> +Calculation rule for queue:
> 
> +All queues are allocated to different processes based on ``proc_num`` 
> and ``proc_id``.
> 
> +Calculation rule for the testpmd to allocate queues to each process:
> 
> +start(queue start id) = proc_id * nb_q / num_procs?
> 
> +end(queue end id) = start + nb_q / num_procs?
> 
> +
> 
> +For example, if testpmp supports 4 Tx and Rx queues the 0~1 for primary
> 
> // [Spelling] testpmp à “testpmd”, change Rx à 4 Rx
> 
> +process the 2~3 for secondary process
> 
> +
> 
> +The number of rings should be a multiple of the number of processes. If
> 
> +not, redundant queues will exist after queues are allocated to
> 
> +processes. After RSS is enabled, packet loss occurs when traffic is 
> sent to all processes at the same time.
> 
> +Some traffic enters redundant queues and cannot be forwarded.
> 
> +
> 
> +All the dev ops is supported in primary process. While secondary
> 
> +process is not permitted to allocate or release shared memory, so some 
> ops are not supported as follows::
> 
> +``dev_configure``
> 
> +``dev_start``
> 
> +``dev_stop``
> 
> +``rx_queue_setup``
> 
> +``tx_queue_setup``
> 
> +``rx_queue_release``
> 
> +``tx_queue_release``
> 
> +
> 
> +So, any command from testpmd which calls those APIs will not be
> 
> +supported in secondary process, like::
> 
> +``port config all rxq|txq|rxd|txd <value>`` ``port config <port_id>
> 
> +rx_offload xxx on/off `` ``port config <port_id> tx_offload xxx
> 
> +on/off`` etc.
> 
> +
> 
> +Flow API is supported, it applies only on its own process on SW side, 
> but all on HW size.
> 
> // [Replace] all à uniformly    [Spelling] size --> side
> 
> +
> 
> +Stats is supported, stats will not change when one quit and start, As
> 
> +they share the same buffer to store the stats. Flow rules are
> 
> +maintained in process level: primary and secondary has its own flow
> 
> +list (but one flow list in HW). The two can see all the queues, so
> 
> +setting the flow rules for the other is OK. But in testpmd primary
> 
> // [Replace] in à if
> 
> +process receives or transmits packets from the queue allocated for 
> secondary process is not permitted, and same for secondary process.
> 
> // [Rephrase] is not permittedà it is not permitted
> 
> +
> 
> +RSS is supported, primary process and secondary process has separate
> 
> +queues to use, RSS will work in their own queues whether primary or 
> secondary process.
> 
> --
> 
> 2.7.4
> 
> Tested-by: Aman Deep Singh <aman.deep.singh at intel.com 
> <mailto:aman.deep.singh at intel.com>>
>