<html>
<head>
<meta http-equiv="Content-Type" content="text/html; charset=us-ascii">
<style type="text/css" style="display:none;"> P {margin-top:0;margin-bottom:0;} </style>
</head>
<body dir="ltr">
<div style="font-family: Calibri, Arial, Helvetica, sans-serif; font-size: 12pt; color: rgb(0, 0, 0); background-color: rgb(255, 255, 255);" class="elementToProof">
Thanks Asaf for your reply.</div>
<div style="font-family: Calibri, Arial, Helvetica, sans-serif; font-size: 12pt; color: rgb(0, 0, 0); background-color: rgb(255, 255, 255);" class="elementToProof">
<br>
</div>
<div style="font-family: Calibri, Arial, Helvetica, sans-serif; font-size: 12pt; color: rgb(0, 0, 0); background-color: rgb(255, 255, 255);" class="elementToProof ContentPasted1 ContentPasted18">
We want to use shared_rxqs between PF and VF_rep. I tried to test this feature with the testpmd shared_rxq fwd. We used the "-<span style="color: rgb(0, 0, 0); background-color: rgb(255, 255, 255); display: inline !important;" class="ContentPasted16">-rxq-share</span>"
option in the command line to run the testpmd. With <span style="color: rgb(0, 0, 0); background-color: rgb(255, 255, 255); display: inline !important;" class="ContentPasted0">server A, we were able to create shared_rxqs between PF and VF_rep0, but with server
B we were not able to create shared_rxqs</span> instead server B created individual RX queues on PF and VF_rep0. We confirmed this by printing the stats of both ports. <span style="color: rgb(0, 0, 0); background-color: rgb(255, 255, 255); display: inline !important;" class="ContentPasted19"> </span></div>
<div style="font-family: Calibri, Arial, Helvetica, sans-serif; font-size: 12pt; color: rgb(0, 0, 0); background-color: rgb(255, 255, 255);" class="elementToProof ContentPasted1 ContentPasted18">
<span class="ContentPasted19" style="margin: 0px; color: rgb(0, 0, 0); background-color: rgb(255, 255, 255); display: inline !important;"><br>
</span></div>
<div style="font-family: Calibri, Arial, Helvetica, sans-serif; font-size: 12pt; color: rgb(0, 0, 0); background-color: rgb(255, 255, 255);" class="elementToProof ContentPasted1 ContentPasted18">
<span class="ContentPasted19" style="margin: 0px; color: rgb(0, 0, 0); background-color: rgb(255, 255, 255); display: inline !important;">As in mlx5 doc "<b>Shared Rx queue: Counters of received packets and bytes number of devices in same share group are same.
Counters of received packets and bytes number of queues in same group and queue ID are same.</b>"</span></div>
<div style="font-family: Calibri, Arial, Helvetica, sans-serif; font-size: 12pt; color: rgb(0, 0, 0); background-color: rgb(255, 255, 255);" class="elementToProof ContentPasted1">
<br>
</div>
<div style="font-family: Calibri, Arial, Helvetica, sans-serif; font-size: 12pt; color: rgb(0, 0, 0); background-color: rgb(255, 255, 255);" class="elementToProof ContentPasted1">
As <span style="color: rgb(0, 0, 0); background-color: rgb(255, 255, 255); display: inline !important;" class="ContentPasted20">PF and VF_rep0</span> should have the same RX stats, by printing port stats server A shows the same stats on <span style="color: rgb(0, 0, 0); background-color: rgb(255, 255, 255); display: inline !important;" class="ContentPasted17">both
PF and VF_rep</span> but server B doesn't. Please review the below test configurations
<span style="color: rgb(0, 0, 0); background-color: rgb(255, 255, 255); display: inline !important;" class="ContentPasted21">
and port stats </span>on both servers.</div>
<div style="font-family: Calibri, Arial, Helvetica, sans-serif; font-size: 12pt; color: rgb(0, 0, 0); background-color: rgb(255, 255, 255);" class="elementToProof">
<br>
</div>
<div style="font-family: Calibri, Arial, Helvetica, sans-serif; font-size: 12pt; color: rgb(0, 0, 0); background-color: rgb(255, 255, 255);" class="elementToProof">
<b>Configurations on SERVER A and SERVER B:</b></div>
<div style="font-family: Calibri, Arial, Helvetica, sans-serif; font-size: 12pt; color: rgb(0, 0, 0); background-color: rgb(255, 255, 255);" class="elementToProof ContentPasted3">
$ sudo ./build_22_07/app/dpdk-testpmd -l 0-1 -n 4 -a 0000:04:00.0,representor=vf[0] -- -i --nb-cores=1 --rxq-share=2<br>
</div>
<div style="font-family: Calibri, Arial, Helvetica, sans-serif; font-size: 12pt; color: rgb(0, 0, 0); background-color: rgb(255, 255, 255);" class="elementToProof ContentPasted3 ContentPasted10 ContentPasted11">
mlx5_common: DevX create q counter set failed errno=22 status=0 syndrome=0<br class="ContentPasted11">
mlx5_common: DevX create q counter set failed errno=22 status=0 syndrome=0<br>
</div>
<div style="font-family: Calibri, Arial, Helvetica, sans-serif; font-size: 12pt; color: rgb(0, 0, 0); background-color: rgb(255, 255, 255);" class="elementToProof ContentPasted3 ContentPasted10 ContentPasted11">
<br>
</div>
<div style="font-family: Calibri, Arial, Helvetica, sans-serif; font-size: 12pt; color: rgb(0, 0, 0); background-color: rgb(255, 255, 255);" class="elementToProof ContentPasted3 ContentPasted10">
testpmd> set fwd shared_rxq <br>
</div>
<div style="font-family: Calibri, Arial, Helvetica, sans-serif; font-size: 12pt; color: rgb(0, 0, 0); background-color: rgb(255, 255, 255);" class="elementToProof ContentPasted3 ContentPasted10">
testpmd> start</div>
<div style="font-family: Calibri, Arial, Helvetica, sans-serif; font-size: 12pt; color: rgb(0, 0, 0); background-color: rgb(255, 255, 255);" class="elementToProof ContentPasted3 ContentPasted10">
<br>
</div>
<div style="font-family: Calibri, Arial, Helvetica, sans-serif; font-size: 12pt; color: rgb(0, 0, 0); background-color: rgb(255, 255, 255);" class="elementToProof ContentPasted3 ContentPasted10">
<b>SEND PACKET FROM SCAPY:</b></div>
<div style="font-family: Calibri, Arial, Helvetica, sans-serif; font-size: 12pt; color: rgb(0, 0, 0); background-color: rgb(255, 255, 255);" class="elementToProof ContentPasted3">
<span style="font-family: Calibri, Arial, Helvetica, sans-serif; font-size: 12pt;">The packet sent from the wire is received on PF but port stats of both were updated correctly:</span><br>
</div>
<div style="font-family: Calibri, Arial, Helvetica, sans-serif; font-size: 12pt; color: rgb(0, 0, 0); background-color: rgb(255, 255, 255);" class="elementToProof ContentPasted3 ContentPasted4">
sendp(Ether()/Dot1Q(vlan=123)/IP()/UDP(),iface='enp130s0f0',count=24)<br>
</div>
<div style="font-family: Calibri, Arial, Helvetica, sans-serif; font-size: 12pt; color: rgb(0, 0, 0); background-color: rgb(255, 255, 255);" class="elementToProof ContentPasted3 ContentPasted4">
<br>
</div>
<div style="font-family: Calibri, Arial, Helvetica, sans-serif; font-size: 12pt; color: rgb(0, 0, 0); background-color: rgb(255, 255, 255);" class="elementToProof ContentPasted3 ContentPasted4">
<b>SHOW STATS ON SERVER A:</b></div>
<div style="font-family: Calibri, Arial, Helvetica, sans-serif; font-size: 12pt; color: rgb(0, 0, 0); background-color: rgb(255, 255, 255);" class="elementToProof ContentPasted3 ContentPasted4 ContentPasted5">
</div>
testpmd> show port stats all
<div class="elementToProof"><br class="ContentPasted12">
</div>
<div class="ContentPasted12"> ######################## NIC statistics for port 0 ########################</div>
<div class="ContentPasted12 elementToProof"> RX-packets: 24 RX-missed: 0 RX-bytes: 1440</div>
<div class="ContentPasted12"> RX-errors: 0</div>
<div class="ContentPasted12"> RX-nombuf: 0 </div>
<div class="ContentPasted12 elementToProof"> TX-packets: 0 TX-errors: 0 TX-bytes: 0</div>
<div><br class="ContentPasted12">
</div>
<div class="ContentPasted12"> Throughput (since last show)</div>
<div class="ContentPasted12 elementToProof"> Rx-pps: 0 Rx-bps: 0</div>
<div class="ContentPasted12"> Tx-pps: 0 Tx-bps: 0</div>
<div class="ContentPasted12"> ############################################################################</div>
<div><br class="ContentPasted12">
</div>
<div class="ContentPasted12"> ######################## NIC statistics for port 1 ########################</div>
<div class="ContentPasted12"> RX-packets: 24 RX-missed: 0 RX-bytes: 1440</div>
<div class="ContentPasted12"> RX-errors: 0</div>
<div class="ContentPasted12"> RX-nombuf: 0 </div>
<div class="ContentPasted12"> TX-packets: 0 TX-errors: 0 TX-bytes: 0</div>
<div><br class="ContentPasted12">
</div>
<div class="ContentPasted12"> Throughput (since last show)</div>
<div class="ContentPasted12"> Rx-pps: 0 Rx-bps: 0</div>
<div class="ContentPasted12"> Tx-pps: 0 Tx-bps: 0</div>
<div class="ContentPasted12"> ############################################################################</div>
<div style="font-family: Calibri, Arial, Helvetica, sans-serif; font-size: 12pt; color: rgb(0, 0, 0); background-color: rgb(255, 255, 255);" class="elementToProof ContentPasted3 ContentPasted4 ContentPasted5 ContentPasted6 ContentPasted7">
<br>
</div>
<div style="font-family: Calibri, Arial, Helvetica, sans-serif; font-size: 12pt; color: rgb(0, 0, 0); background-color: rgb(255, 255, 255);" class="elementToProof ContentPasted3 ContentPasted4 ContentPasted5 ContentPasted6 ContentPasted7">
<b><span style="color: rgb(0, 0, 0); background-color: rgb(255, 255, 255); display: inline !important;" class="ContentPasted13"><b style="color: rgb(0, 0, 0);" class="ContentPasted15">SHOW STATS ON SERVER B:</b></span><br>
</b></div>
<div style="font-family: Calibri, Arial, Helvetica, sans-serif; font-size: 12pt; color: rgb(0, 0, 0); background-color: rgb(255, 255, 255);" class="elementToProof">
<br>
</div>
<div style="font-family: Calibri, Arial, Helvetica, sans-serif; font-size: 12pt; color: rgb(0, 0, 0); background-color: rgb(255, 255, 255);" class="elementToProof ContentPasted14">
testpmd> show port stats all
<div><br class="ContentPasted14">
</div>
<div class="ContentPasted14"> ######################## NIC statistics for port 0 ########################</div>
<div class="ContentPasted14"> RX-packets: 24 RX-missed: 0 RX-bytes: 1440</div>
<div class="ContentPasted14"> RX-errors: 0</div>
<div class="ContentPasted14"> RX-nombuf: 0 </div>
<div class="ContentPasted14"> TX-packets: 0 TX-errors: 0 TX-bytes: 0</div>
<div><br class="ContentPasted14">
</div>
<div class="ContentPasted14"> Throughput (since last show)</div>
<div class="ContentPasted14"> Rx-pps: 1 Rx-bps: 808</div>
<div class="ContentPasted14"> Tx-pps: 0 Tx-bps: 0</div>
<div class="ContentPasted14"> ############################################################################</div>
<div><br class="ContentPasted14">
</div>
<div class="ContentPasted14"> ######################## NIC statistics for port 1 ########################</div>
<div class="ContentPasted14"> RX-packets: 0 RX-missed: 0 RX-bytes: 0</div>
<div class="ContentPasted14"> RX-errors: 0</div>
<div class="ContentPasted14"> RX-nombuf: 0 </div>
<div class="ContentPasted14"> TX-packets: 0 TX-errors: 0 TX-bytes: 0</div>
<div><br class="ContentPasted14">
</div>
<div class="ContentPasted14"> Throughput (since last show)</div>
<div class="ContentPasted14"> Rx-pps: 0 Rx-bps: 0</div>
<div class="ContentPasted14"> Tx-pps: 0 Tx-bps: 0</div>
<div class="ContentPasted14"> ############################################################################</div>
<div class="ContentPasted14"><br>
</div>
</div>
<div>
<div id="Signature">
<div>
<div style="font-family: Calibri, Arial, Helvetica, sans-serif; font-size: 12pt; color: rgb(0, 0, 0);">
Regards,</div>
<div style="font-family: Calibri, Arial, Helvetica, sans-serif; font-size: 12pt; color: rgb(0, 0, 0);">
Haider</div>
</div>
</div>
</div>
<div id="appendonsend"></div>
<hr style="display:inline-block;width:98%" tabindex="-1">
<div id="divRplyFwdMsg" dir="ltr"><font face="Calibri, sans-serif" style="font-size:11pt" color="#000000"><b>From:</b> Asaf Penso <asafp@nvidia.com><br>
<b>Sent:</b> Thursday, October 6, 2022 11:44 AM<br>
<b>To:</b> Haider Ali <haider@dreambigsemi.com>; users <users@dpdk.org><br>
<b>Subject:</b> Re: mlx5 - shared_rxq not working on some Connect x6 DX</font>
<div> </div>
</div>
<div>
<div dir="auto">Hello Haider,</div>
<div dir="auto">Besides the info print issue, can you describe what is the use case?</div>
<div dir="auto">What do you try to achieve and whether you see any issue?</div>
<div><br>
</div>
<div dir="auto" id="x_ms-outlook-mobile-signature">
<div>Regards,</div>
<div dir="auto">Asaf Penso</div>
</div>
<hr tabindex="-1" style="display:inline-block; width:98%">
<div id="x_divRplyFwdMsg" dir="ltr"><font face="Calibri, sans-serif" color="#000000" style="font-size:11pt"><b>From:</b> Haider Ali <haider@dreambigsemi.com><br>
<b>Sent:</b> Tuesday, October 4, 2022 4:24:03 PM<br>
<b>To:</b> users <users@dpdk.org><br>
<b>Cc:</b> Asaf Penso <asafp@nvidia.com><br>
<b>Subject:</b> mlx5 - shared_rxq not working on some Connect x6 DX</font>
<div> </div>
</div>
<style type="text/css" style="display:none">
<!--
p
{margin-top:0;
margin-bottom:0}
-->
</style>
<div dir="ltr">
<div class="x_x_elementToProof x_x_ContentPasted7" style="font-family:Calibri,Arial,Helvetica,sans-serif; font-size:12pt; color:rgb(0,0,0); background-color:rgb(255,255,255)">
Hi,
<div><br class="x_x_ContentPasted7">
</div>
<div>I used two servers A and B. Server A has <span class="x_x_ContentPasted8" style="color:rgb(0,0,0); background-color:rgb(255,255,255); display:inline!important">Connect x5 and x6 DX cards while server B has <span class="x_x_ContentPasted9" style="color:rgb(0,0,0); background-color:rgb(255,255,255); display:inline!important">Connect
x6 DX card.</span></span></div>
<div><span class="x_x_ContentPasted8" style="color:rgb(0,0,0); background-color:rgb(255,255,255); display:inline!important"><span class="x_x_ContentPasted9" style="color:rgb(0,0,0); background-color:rgb(255,255,255); display:inline!important"><br>
</span></span></div>
<div class="x_x_ContentPasted7">I have checked with Connect x5 and x6 DX cards on "server A" and I am able to get below rxq_share device capability:</div>
<div><br class="x_x_ContentPasted7">
</div>
<div class="x_x_ContentPasted7">testpmd> show port info all</div>
<div><br class="x_x_ContentPasted7">
</div>
<div class="x_x_ContentPasted7">Device capabilities: 0x14( <b>RXQ_SHARE</b> FLOW_SHARED_OBJECT_KEEP )</div>
<div><br class="x_x_ContentPasted7">
</div>
<div class="x_x_ContentPasted7">Server A setting:</div>
<div class="x_x_ContentPasted7"># ofed_info -s</div>
<div class="x_x_ContentPasted7"><b>MLNX_OFED_LINUX-5.4-3.1.0.0:</b></div>
<div><br class="x_x_ContentPasted7">
</div>
<div class="x_x_ContentPasted7"># ethtool -i enp129s0f0</div>
<div class="x_x_ContentPasted7">driver: mlx5_core</div>
<div class="x_x_ContentPasted7">version: 5.4-3.1.0</div>
<div class="x_x_ContentPasted7"><b>firmware-version: 16.32.1010 (MT_0000000080)</b></div>
<div class="x_x_ContentPasted7">expansion-rom-version:</div>
<div class="x_x_ContentPasted7">bus-info: 0000:81:00.0</div>
<div class="x_x_ContentPasted7">supports-statistics: yes</div>
<div class="x_x_ContentPasted7">supports-test: yes</div>
<div class="x_x_ContentPasted7">supports-eeprom-access: no</div>
<div class="x_x_ContentPasted7">supports-register-dump: no</div>
<div class="x_x_ContentPasted7">supports-priv-flags: yes</div>
<div><br class="x_x_ContentPasted7">
</div>
<div><br class="x_x_ContentPasted7">
</div>
<div class="x_x_ContentPasted7">But when I moved to server B with another Connect x6 DX card, I could not see this capability.</div>
<div><br class="x_x_ContentPasted7">
</div>
<div><span class="x_x_ContentPasted7 x_x_ContentPasted10" style="margin:0px; color:rgb(0,0,0)">testpmd> show port info all</span>
<div style="margin:0px; color:rgb(0,0,0)"><br class="x_x_ContentPasted7 x_x_ContentPasted10">
</div>
</div>
<div><span class="x_x_ContentPasted7 x_x_ContentPasted10" style="margin:0px; color:rgb(0,0,0)">Device capabilities: 0x14(<span class="x_x_ContentPasted10"> </span>FLOW_SHARED_OBJECT_KEEP )</span><br>
</div>
<div><span class="x_x_ContentPasted7 x_x_ContentPasted10" style="margin:0px; color:rgb(0,0,0)"><br>
</span></div>
<div class="x_x_ContentPasted7">Server B Settings:</div>
<div class="x_x_ContentPasted7"># ofed_info -s</div>
<div class="x_x_ContentPasted7"><b>MLNX_OFED_LINUX-5.4-3.5.8.0:</b></div>
<div><br class="x_x_ContentPasted7">
</div>
<div class="x_x_ContentPasted7"># ethtool -i enp132s0f0</div>
<div class="x_x_ContentPasted7">driver: mlx5_core</div>
<div class="x_x_ContentPasted7">version: 5.4-3.5.8</div>
<div class="x_x_ContentPasted7"><b>firmware-version: 22.31.1014 (MT_0000000436)</b></div>
<div class="x_x_ContentPasted7">expansion-rom-version:</div>
<div class="x_x_ContentPasted7">bus-info: 0000:84:00.0</div>
<div class="x_x_ContentPasted7">supports-statistics: yes</div>
<div class="x_x_ContentPasted7">supports-test: yes</div>
<div class="x_x_ContentPasted7">supports-eeprom-access: no</div>
<div class="x_x_ContentPasted7">supports-register-dump: no</div>
<div class="x_x_ContentPasted7">supports-priv-flags: yes</div>
<div><br class="x_x_ContentPasted7">
</div>
<div><br class="x_x_ContentPasted7">
</div>
<div class="x_x_ContentPasted7"><b>Although server B has higher versions of OFED and firmware, my question is do I need to enable/disable firmware settings? Or are there any other configurations we need to apply?</b></div>
<div class="x_x_ContentPasted7"><b><br>
</b></div>
<div class="x_x_ContentPasted7">Regards,</div>
Haider<br>
</div>
</div>
</div>
</body>
</html>