[dpdk-dev] [PATCH v11] app/testpmd: support multi-process
Ferruh Yigit
ferruh.yigit at intel.com
Mon Apr 19 15:42:25 CEST 2021
On 4/19/2021 2:03 AM, Min Hu (Connor) wrote:
> This patch adds multi-process support for testpmd.
> The test cmd example as follows:
> the primary cmd:
> ./dpdk-testpmd -a xxx --proc-type=auto -l 0-1 -- -i \
> --rxq=4 --txq=4 --num-procs=2 --proc-id=0
>
> the secondary cmd:
> ./dpdk-testpmd -a xxx --proc-type=auto -l 2-3 -- -i \
> --rxq=4 --txq=4 --num-procs=2 --proc-id=1
>
Hi Connor,
I put some comments on the documentation, can you please check them?
Meanwhile, John, Marko can you please help on the doc review?
Thanks,
ferruh
> Signed-off-by: Min Hu (Connor) <humin29 at huawei.com>
> Signed-off-by: Lijun Ou <oulijun at huawei.com>
> Acked-by: Xiaoyun Li <xiaoyun.li at intel.com>
> Acked-by: Ajit Khaparde <ajit.khaparde at broadcom.com>
> Reviewed-by: Ferruh Yigit <ferruh.yigit at intel.com>
<...>
> @@ -2626,16 +2701,16 @@ start_port(portid_t pid)
> cnt_pi++;
>
> /* start port */
> - diag = rte_eth_dev_start(pi);
> + diag = eth_dev_start_mp(pi);
> if (diag < 0) {
> printf("Fail to start port %d: %s\n", pi,
> rte_strerror(-diag));
>
> /* Fail to setup rx queue, return */
> if (rte_atomic16_cmpset(&(port->port_status),
> - RTE_PORT_HANDLING, RTE_PORT_STOPPED) == 0)
> - printf("Port %d can not be set back to "
> - "stopped\n", pi);
> + RTE_PORT_HANDLING, RTE_PORT_STOPPED) == 0)
> + printf("Port %d can not be set back to stopped\n",
> + pi);
Indentation is wrong.
<...>
> --- a/doc/guides/testpmd_app_ug/run_app.rst
> +++ b/doc/guides/testpmd_app_ug/run_app.rst
> @@ -543,3 +543,89 @@ The command line options are:
> bit 1 - two hairpin ports paired
> bit 0 - two hairpin ports loop
> The default value is 0. Hairpin will use single port mode and implicit Tx flow mode.
> +
> +
> +Testpmd Support Multi Process Command-line Options
> +--------------------------------------------------
What do you think making this a sub-section of the "Testpmd Command-line Options"?
And the section name can be "Testpmd Multi-Process Command-line Options"
> +
> +The following are the command-line options for the testpmd applications(support
Space before '('.
> +multi process).They must be separated from the EAL options, shown in the previous
> +section, with a ``--`` separator:
If this becomes sub-section of the "Testpmd Command-line Options", above will
become duplication, and I think it can be simplified as:
"The following are the command-line options for testpmd multi-process support:"
> +
> +.. code-block:: console
> +
> + primary process:
> + sudo ./dpdk-testpmd -a xxx --proc-type=auto -l 0-1 -- -i --rxq=4 --txq=4 \
> + --num-procs=2 --proc-id=0
> +
> + secondary process:
> + sudo ./dpdk-testpmd -a xxx --proc-type=auto -l 2-3 -- -i --rxq=4 --txq=4 \
> + --num-procs=2 --proc-id=1
> +
> +The command line options are:
> +
> +* ``--rxq=N``
> +
> + Set the number of Rx queues per port to N. N is the sum of queues used by primary
> + and secondary process. Primary process and secondary process should have separate
> + queues, and each should occupy at least one queue. Where N should be the multiple
> + of number of processes.
> +
> +* ``--txq=N``
> +
> + Set the number of Tx queues per port to N. N is the sum of queues used by primary
> + and secondary process. Primary process and secondary process should have separate
> + queues, and each should occupy at least one queue. Where N should be the multiple
> + of number of processes.
> +
> +* ``--num-procs=N``
> +
> + The number of processes which will be used.
> +
> +* ``--proc-id=id``
> +
> + The id of the current process (id < num-procs). id should be different in primary
> + process and secondary process.
> +
The famous question, does it start from '0' or '1'?
> +Calculation rule for queue:
> +All queues are allocated to different processes based on proc_num and proc_id.
Can highlight proc_num and proc_id, as ``proc_num`` and ``proc_id``.
> +Calculation rule for the Testpmd to allocate queues to each process:
Not sure if testpmd should start with uppercase.
> +start(queue start id) = proc_id * nb_q / num_procs;
> +end(queue end id) = start + nb_q / num_procs;
> +
Can you put above into a code-block.
> +For example, if supports 4 txq and rxq
".., if testpmp supports 4 Tx and Rx queues"
> +the 0~1 for primary process
> +the 2~3 for secondary process
> +
> +The number of rings should be a multiple of the number of processes. If not,
> +redundant queues will exist after queues are allocated to processes. After RSS is
> +enabled, packet loss occurs when traffic is sent to all processes at the same time.
> +Some traffic enters redundant queues and cannot be forwarded.
> +
> +Most dev ops is supported in primary and secondary process. While secondary process
"Most of the device operations are supported in ..."?
"While the secondary process ..."?
> +is not permitted to allocate or release shared memory, so some ops are not supported
> +as follows:
> +``dev_configure``
> +``dev_start``
> +``dev_stop``
> +``rx_queue_setup``
> +``tx_queue_setup``
> +``rx_queue_release``
> +``tx_queue_release``
> +
This list displayed as single line in the html output, if the intention is to
have them listed as been in the source, may need following update:
-as follows:
+as follows::
+
> +So, any command from testpmd which calls those APIs will not be supported in secondary
> +process, like:
> +``port config all rxq|txq|rxd|txd <value>``
> +``port config <port_id> rx_offload xxx on/off ``
> +``port config <port_id> tx_offload xxx on/off``
> +etc.
Same here for the list view.
> +
> +RTE_FLOW supported, it applies only on its own process on SW side, but all on HW size.
You may use "Flow API", instead of RTE_FLOW, so it becomes: "Flow API is supported."
> +stats supported, stats will not change when one quit and start, As they share the same
> +buffer to store the stats.
The above stats related sentences looks like squeezed within the flow API
related description, it may be good to move it.
> Flow rules are maintained in process level: primary and secondary
> +has its own flow list(but one flow list in HW). The two can see all the queues, so setting
Space before '('.
> +the flow rules for the other is OK. Of course, io(receive or transmit packets) in the queue
> +from others is not permitted.
I understand what you mean, but wording can be improved, can you please try to
reword it?
> +
> +RSS supported, Primary process and secondary process has separate queues to use, RSS
"RSS is supported"? And start 'primary' with lowercase.
> +will work in their own queues whether primary and secondary process.
>
There is a chance that when RSS is enabled, the packet may end up in other
process' queues?
More information about the dev
mailing list