<html>
<head>
<meta http-equiv="Content-Type" content="text/html; charset=koi8-r">
</head>
<body>
<div dir="auto">Hello Dmitry,</div>
<div dir="auto"><br>
</div>
<div dir="auto">Could you please paste the testpmd commands per each experiment?</div>
<div dir="auto"><br>
</div>
<div dir="auto">Also, have you looked into dpdk.org performance report to see how to tune for best results?</div>
<div><br>
</div>
<div id="ms-outlook-mobile-signature" dir="auto">
<div>Regards,</div>
<div dir="auto">Asaf Penso</div>
</div>
<hr style="display:inline-block;width:98%" tabindex="-1">
<div id="divRplyFwdMsg" dir="ltr"><font face="Calibri, sans-serif" style="font-size:11pt" color="#000000"><b>From:</b> Дмитрий Степанов <stepanov.dmit@gmail.com><br>
<b>Sent:</b> Friday, February 18, 2022 9:32:59 AM<br>
<b>To:</b> users@dpdk.org <users@dpdk.org><br>
<b>Subject:</b> Mellanox performance degradation with more than 12 lcores</font>
<div> </div>
</div>
<div>
<div dir="ltr">
<div>Hi folks!</div>
<div><br>
</div>
I'm using Mellanox ConnectX-6 Dx EN adapter card (100GbE; Dual-port QSFP56; PCIe 4.0/3.0 x16) with DPDK 21.11 on a server with AMD EPYC 7702 64-Core Processor (NUMA system with 2 sockets). Hyperthreading is turned off.<br>
I'm testing the maximum receive throughput I can get from a single port using testpmd utility (shipped with dpdk). My generator produces random UDP packets with zero payload length.
<br>
<br>
I get the maximum performance using 8-12 lcores (overall 120-125Mpps on receive path of single port):<br>
<br>
numactl -N 1 -m 1 /opt/dpdk-21.11/build/app/dpdk-testpmd -l 64-127 -n 4 -a 0000:c1:00.0 -- --stats-period 1 --nb-cores=12 --rxq=12 --txq=12 --rxd=512<br>
<br>
With more than 12 lcores overall receive performance reduces. With 16-32 lcores I get 100-110 Mpps, and I get a significant performance fall with 33 lcores - 84Mpps. With 63 cores I get even 35Mpps overall receive performance.<br>
<br>
Are there any limitations on the total number of receive queues (total lcores) that can handle a single port on a given NIC?<br>
<div><br>
</div>
<div>Thanks,<br>
Dmitriy Stepanov<br>
</div>
</div>
</div>
</body>
</html>