[PATCH v3] app/testpmd: fix secondary process not forwarding

[He, ShiyangX]

>-----Original Message-----
>From: Ferruh Yigit <ferruh.yigit at amd.com>
>Sent: Monday, March 6, 2023 11:06 PM
>To: He, ShiyangX <shiyangx.he at intel.com>; dev at dpdk.org
>Cc: Zhou, YidingX <yidingx.zhou at intel.com>; stable at dpdk.org; Zhang, Yuying
><yuying.zhang at intel.com>; Singh, Aman Deep
><aman.deep.singh at intel.com>; Burakov, Anatoly
><anatoly.burakov at intel.com>; Matan Azrad <matan at nvidia.com>; Dmitry
>Kozlyuk <dmitry.kozliuk at gmail.com>
>Subject: Re: [PATCH v3] app/testpmd: fix secondary process not forwarding
>
>On 2/23/2023 2:41 PM, Shiyang He wrote:
>> Under multi-process scenario, the secondary process gets queue state
>> from the wrong location (the global variable 'ports'). Therefore, the
>> secondary process can not forward since "stream_init" is not called.
>>
>> This commit fixes the issue by calling 'rte_eth_rx/tx_queue_info_get'
>> to get queue state from shared memory.
>>
>> Fixes: 3c4426db54fc ("app/testpmd: do not poll stopped queues")
>> Cc: stable at dpdk.org
>>
>> Signed-off-by: Shiyang He <shiyangx.he at intel.com>
>> Acked-by: Yuying Zhang <yuying.zhang at intel.com>
>>
>> v3: Add return value description
>> ---
>>  app/test-pmd/testpmd.c | 45
>> ++++++++++++++++++++++++++++++++++++++++--
>>  1 file changed, 43 insertions(+), 2 deletions(-)
>>
>> diff --git a/app/test-pmd/testpmd.c b/app/test-pmd/testpmd.c index
>> 0c14325b8d..a050472aea 100644
>> --- a/app/test-pmd/testpmd.c
>> +++ b/app/test-pmd/testpmd.c
>> @@ -2418,9 +2418,50 @@ start_packet_forwarding(int with_tx_first)
>>  	if (!pkt_fwd_shared_rxq_check())
>>  		return;
>>
>> -	if (stream_init != NULL)
>> -		for (i = 0; i < cur_fwd_config.nb_fwd_streams; i++)
>> +	if (stream_init != NULL) {
>> +		for (i = 0; i < cur_fwd_config.nb_fwd_streams; i++) {
>> +			if (rte_eal_process_type() == RTE_PROC_SECONDARY)
>{
>> +				struct fwd_stream *fs = fwd_streams[i];
>> +				struct rte_eth_rxq_info rx_qinfo;
>> +				struct rte_eth_txq_info tx_qinfo;
>> +				int32_t rc;
>> +				rc = rte_eth_rx_queue_info_get(fs->rx_port,
>> +						fs->rx_queue, &rx_qinfo);
>> +				if (rc == 0) {
>> +					ports[fs->rx_port].rxq[fs-
>>rx_queue].state =
>> +						rx_qinfo.queue_state;
>> +				} else if (rc == -ENOTSUP) {
>> +					/* Set the rxq state to
>RTE_ETH_QUEUE_STATE_STARTED
>> +					 * to ensure that the PMDs do not
>implement
>> +					 * rte_eth_rx_queue_info_get can
>forward.
>> +					 */
>> +					ports[fs->rx_port].rxq[fs-
>>rx_queue].state =
>> +
>	RTE_ETH_QUEUE_STATE_STARTED;
>> +				} else {
>> +					TESTPMD_LOG(WARNING,
>> +						"Failed to get rx queue
>info\n");
>> +				}
>> +
>> +				rc = rte_eth_tx_queue_info_get(fs->tx_port,
>> +						fs->tx_queue, &tx_qinfo);
>> +				if (rc == 0) {
>> +					ports[fs->tx_port].txq[fs-
>>tx_queue].state =
>> +						tx_qinfo.queue_state;
>> +				} else if (rc == -ENOTSUP) {
>> +					/* Set the txq state to
>RTE_ETH_QUEUE_STATE_STARTED
>> +					 * to ensure that the PMDs do not
>implement
>> +					 * rte_eth_tx_queue_info_get can
>forward.
>> +					 */
>> +					ports[fs->tx_port].txq[fs-
>>tx_queue].state =
>> +
>	RTE_ETH_QUEUE_STATE_STARTED;
>> +				} else {
>> +					TESTPMD_LOG(WARNING,
>> +						"Failed to get tx queue
>info\n");
>> +				}
>> +			}
>>  			stream_init(fwd_streams[i]);
>> +		}
>> +	}
>>
>
>
>Testpmd duplicates some dpdk/ethdev state/config in application level, and
>this can bite in multiple cases, as it is happening here.
>
>I am not sure if this was a design decision, but I think instead of testpmd
>storing ethdev related state/config in application level, it should store only
>application level state/config, and when ethdev related state/config is
>required app should get it directly from ethdev.
>
>It may be too late already for testpmd, there is a mixed usage, but I am for
>preferring this approach when there is an opportunity.
>
>
>
>For above issue, why queue state needs to be stored in application level 'port'
>variable?
>Where is this queue state used?
>
>Can it work to get queue state directly from ethdev where this state is used,
>instead of storing it in the 'port' variable in advance?
>
>And perhaps testpmd 'port' variable can be updated there, both for primary
>and secondary, for backward compatibility (other existing users of this queue
>state).
>
>What do you think?

Thanks for your comments!

It is an effective method to get queue state directly from ethdev where this state is used.
I also don't know the design meaning of the 'ports' variable. If modification is needed,
a higher level of design and more work are required.

As a bug fix, apart from extracting the code block into a function, is the solution feasible?