<div dir="ltr">Hi All,<div><br></div><div>I'm trying to deploy a l2fwd application on VMs to test it. </div><div>I have a 1Gbps link speed NIC, when I run a simple l2fwd application able to achieve 1gbps traffic without any loss of packet. I have modified the l2fwd application a bit.</div><div>I have two l2fwd applications namely P1 and P2. P1 receives the external traffic and sends it to P2 and P2 simply changes the mac address and sends it back to P1. Finally P1 sends the packet out to the external network.</div><div>With the above setup I'm able to achieve only 330Mbps traffic without packet loss. </div><div><br></div><div>I'm trying to understand how the internal communication happens between the P1 and P2 process, does it use PCI bus for communication? </div><div>Why is there a drop in tps and observe packet loss?</div><div><br></div><div>The setup details are given below.</div><div><span id="gmail-docs-internal-guid-12c123e9-7fff-273c-7d69-c84e2fb1195a"><p dir="ltr" style="line-height:1.295;margin-top:0pt;margin-bottom:8pt"><span style="font-size:11pt;font-family:Calibri,sans-serif;color:rgb(0,0,0);background-color:transparent;font-variant-numeric:normal;font-variant-east-asian:normal;vertical-align:baseline;white-space:pre-wrap">The minimum hardware requirements to use UPF are as follows:</span></p><div dir="ltr" style="" align="left"><table style="border:none;border-collapse:collapse"><colgroup><col width="198"><col width="134"><col width="134"><col width="160"></colgroup><tbody><tr style="height:0pt"><td style="border-width:0.5pt;border-style:solid;border-color:rgb(0,0,0);vertical-align:top;background-color:rgb(204,204,204);padding:0pt 5.4pt;overflow:hidden"><p dir="ltr" style="line-height:1.2;text-align:center;margin-top:2pt;margin-bottom:2pt"><span style="font-size:12pt;font-family:Arial;color:rgb(0,0,0);background-color:transparent;font-weight:700;font-variant-numeric:normal;font-variant-east-asian:normal;vertical-align:baseline;white-space:pre-wrap">Hardware</span></p></td><td colspan="3" style="border-width:0.5pt;border-style:solid;border-color:rgb(0,0,0);vertical-align:top;background-color:rgb(204,204,204);padding:0pt 5.4pt;overflow:hidden"><p dir="ltr" style="line-height:1.2;text-align:center;margin-top:2pt;margin-bottom:2pt"><span style="font-size:12pt;font-family:Arial;color:rgb(0,0,0);background-color:transparent;font-weight:700;font-variant-numeric:normal;font-variant-east-asian:normal;vertical-align:baseline;white-space:pre-wrap">Requirement Details</span></p></td></tr><tr style="height:0pt"><td style="border-width:0.5pt;border-style:solid;border-color:rgb(0,0,0);vertical-align:top;padding:0pt 5.4pt;overflow:hidden"><p dir="ltr" style="line-height:1.2;margin-top:2pt;margin-bottom:2pt"><span style="font-size:10pt;font-family:Arial;color:rgb(0,0,0);background-color:transparent;font-variant-numeric:normal;font-variant-east-asian:normal;vertical-align:baseline;white-space:pre-wrap">Machine Type</span></p></td><td colspan="3" style="border-width:0.5pt;border-style:solid;border-color:rgb(0,0,0);vertical-align:top;padding:0pt 5.4pt;overflow:hidden"><p dir="ltr" style="line-height:1.2;margin-top:2pt;margin-bottom:2pt"><span style="font-size:10pt;font-family:Arial;color:rgb(0,0,0);background-color:transparent;font-variant-numeric:normal;font-variant-east-asian:normal;vertical-align:baseline;white-space:pre-wrap">Virtual Machine</span></p></td></tr><tr style="height:0pt"><td style="border-width:0.5pt;border-style:solid;border-color:rgb(0,0,0);vertical-align:top;padding:0pt 5.4pt;overflow:hidden"><p dir="ltr" style="line-height:1.2;margin-top:2pt;margin-bottom:2pt"><span style="font-size:10pt;font-family:Arial;color:rgb(0,0,0);background-color:transparent;font-variant-numeric:normal;font-variant-east-asian:normal;vertical-align:baseline;white-space:pre-wrap">Host Kernel </span></p></td><td colspan="3" style="border-width:0.5pt;border-style:solid;border-color:rgb(0,0,0);vertical-align:top;padding:0pt 5.4pt;overflow:hidden"><p dir="ltr" style="line-height:1.2;margin-top:2pt;margin-bottom:2pt"><span style="font-size:10pt;font-family:Arial;color:rgb(0,0,0);background-color:transparent;font-variant-numeric:normal;font-variant-east-asian:normal;vertical-align:baseline;white-space:pre-wrap">3.10.0-1160.el7.x86_64</span></p></td></tr><tr style="height:0pt"><td style="border-width:0.5pt;border-style:solid;border-color:rgb(0,0,0);vertical-align:top;padding:0pt 5.4pt;overflow:hidden"><p dir="ltr" style="line-height:1.2;margin-top:2pt;margin-bottom:2pt"><span style="font-size:10pt;font-family:Arial;color:rgb(0,0,0);background-color:transparent;font-variant-numeric:normal;font-variant-east-asian:normal;vertical-align:baseline;white-space:pre-wrap">CPU Architecture</span></p><p dir="ltr" style="line-height:1.2;margin-top:2pt;margin-bottom:2pt"><span style="font-size:10pt;font-family:Arial;color:rgb(0,0,0);background-color:transparent;font-variant-numeric:normal;font-variant-east-asian:normal;vertical-align:baseline;white-space:pre-wrap">(Important)</span></p></td><td colspan="3" style="border-width:0.5pt;border-style:solid;border-color:rgb(0,0,0);vertical-align:top;padding:0pt 5.4pt;overflow:hidden"><p dir="ltr" style="line-height:1.2;margin-top:2pt;margin-bottom:2pt"><span style="font-size:10pt;font-family:Arial;color:rgb(0,0,0);background-color:transparent;font-variant-numeric:normal;font-variant-east-asian:normal;vertical-align:baseline;white-space:pre-wrap">x86_64 should support Intel SSE, VT-x, AVX</span></p><p dir="ltr" style="line-height:1.2;margin-top:2pt;margin-bottom:2pt"><span style="font-size:10pt;font-family:Arial;color:rgb(0,0,0);background-color:transparent;font-variant-numeric:normal;font-variant-east-asian:normal;vertical-align:baseline;white-space:pre-wrap">At least Intel Skylake or later gen</span></p></td></tr><tr style="height:0pt"><td style="border-width:0.5pt;border-style:solid;border-color:rgb(0,0,0);vertical-align:top;padding:0pt 5.4pt;overflow:hidden"><p dir="ltr" style="line-height:1.2;margin-top:2pt;margin-bottom:2pt"><span style="font-size:10pt;font-family:Arial;color:rgb(0,0,0);background-color:transparent;font-variant-numeric:normal;font-variant-east-asian:normal;vertical-align:baseline;white-space:pre-wrap">Host OS</span></p></td><td colspan="3" style="border-width:0.5pt;border-style:solid;border-color:rgb(0,0,0);vertical-align:top;padding:0pt 5.4pt;overflow:hidden"><p dir="ltr" style="line-height:1.2;margin-top:2pt;margin-bottom:2pt"><span style="font-size:10pt;font-family:Arial;color:rgb(0,0,0);background-color:transparent;font-variant-numeric:normal;font-variant-east-asian:normal;vertical-align:baseline;white-space:pre-wrap">Centos</span></p></td></tr><tr style="height:0pt"><td style="border-width:0.5pt;border-style:solid;border-color:rgb(0,0,0);vertical-align:top;padding:0pt 5.4pt;overflow:hidden"><p dir="ltr" style="line-height:1.2;margin-top:2pt;margin-bottom:2pt"><span style="font-size:10pt;font-family:Arial;color:rgb(0,0,0);background-color:transparent;font-variant-numeric:normal;font-variant-east-asian:normal;vertical-align:baseline;white-space:pre-wrap">Host Interfaces</span></p></td><td colspan="3" style="border-width:0.5pt;border-style:solid;border-color:rgb(0,0,0);vertical-align:top;padding:0pt 5.4pt;overflow:hidden"><p dir="ltr" style="line-height:1.2;margin-top:2pt;margin-bottom:2pt"><span style="font-size:10pt;font-family:Arial;color:rgb(0,0,0);background-color:transparent;font-variant-numeric:normal;font-variant-east-asian:normal;vertical-align:baseline;white-space:pre-wrap">At least 3 Physical Interfaces</span></p></td></tr><tr style="height:0pt"><td style="border-width:0.5pt;border-style:solid;border-color:rgb(0,0,0);vertical-align:top;padding:0pt 5.4pt;overflow:hidden"><p dir="ltr" style="line-height:1.2;margin-top:2pt;margin-bottom:2pt"><span style="font-size:10pt;font-family:Arial;color:rgb(0,0,0);background-color:transparent;font-variant-numeric:normal;font-variant-east-asian:normal;vertical-align:baseline;white-space:pre-wrap">VM Manager</span></p></td><td colspan="3" style="border-width:0.5pt;border-style:solid;border-color:rgb(0,0,0);vertical-align:top;padding:0pt 5.4pt;overflow:hidden"><p dir="ltr" style="line-height:1.2;margin-top:2pt;margin-bottom:2pt"><span style="font-size:10pt;font-family:Arial;color:rgb(0,0,0);background-color:transparent;font-variant-numeric:normal;font-variant-east-asian:normal;vertical-align:baseline;white-space:pre-wrap">QEMU/KVM minimum V 2.2.1</span></p></td></tr><tr style="height:0pt"><td style="border-width:0.5pt;border-style:solid;border-color:rgb(0,0,0);vertical-align:top;padding:0pt 5.4pt;overflow:hidden"><p dir="ltr" style="line-height:1.2;margin-top:2pt;margin-bottom:2pt"><span style="font-size:10pt;font-family:Arial;color:rgb(0,0,0);background-color:transparent;font-variant-numeric:normal;font-variant-east-asian:normal;vertical-align:baseline;white-space:pre-wrap">VM OS</span></p></td><td colspan="3" style="border-width:0.5pt;border-style:solid;border-color:rgb(0,0,0);vertical-align:top;padding:0pt 5.4pt;overflow:hidden"><p dir="ltr" style="line-height:1.2;margin-top:2pt;margin-bottom:2pt"><span style="font-size:10pt;font-family:Arial;color:rgb(0,0,0);background-color:transparent;font-variant-numeric:normal;font-variant-east-asian:normal;vertical-align:baseline;white-space:pre-wrap">CentOS Linux release 7.9 (Core)</span></p></td></tr><tr style="height:0pt"><td style="border-width:0.5pt;border-style:solid;border-color:rgb(0,0,0);vertical-align:top;padding:0pt 5.4pt;overflow:hidden"><p dir="ltr" style="line-height:1.2;margin-top:2pt;margin-bottom:2pt"><span style="font-size:10pt;font-family:Arial;color:rgb(0,0,0);background-color:transparent;font-variant-numeric:normal;font-variant-east-asian:normal;vertical-align:baseline;white-space:pre-wrap">VM Configuration</span></p></td><td colspan="3" style="border-width:0.5pt;border-style:solid;border-color:rgb(0,0,0);vertical-align:top;padding:0pt 5.4pt;overflow:hidden"><p dir="ltr" style="line-height:1.2;margin-top:2pt;margin-bottom:2pt"><span style="font-size:10pt;font-family:Arial;color:rgb(0,0,0);background-color:transparent;font-variant-numeric:normal;font-variant-east-asian:normal;vertical-align:baseline;white-space:pre-wrap">Minimum Memory 8 GB</span></p><p dir="ltr" style="line-height:1.2;margin-top:2pt;margin-bottom:2pt"><span style="font-size:10pt;font-family:Arial;color:rgb(0,0,0);background-color:transparent;font-variant-numeric:normal;font-variant-east-asian:normal;vertical-align:baseline;white-space:pre-wrap">Minimum 8 core CPUs</span></p><p dir="ltr" style="line-height:1.2;margin-top:2pt;margin-bottom:2pt"><span style="font-size:10pt;font-family:Arial;color:rgb(0,0,0);background-color:transparent;font-variant-numeric:normal;font-variant-east-asian:normal;vertical-align:baseline;white-space:pre-wrap">Minimum 10 GB disk space</span></p></td></tr><tr style="height:10pt"><td rowspan="9" style="border-width:0.5pt;border-style:solid;border-color:rgb(0,0,0);vertical-align:top;padding:0pt 5.4pt;overflow:hidden"><p dir="ltr" style="line-height:1.2;margin-top:2pt;margin-bottom:2pt"><span style="font-size:10pt;font-family:Arial;color:rgb(0,0,0);background-color:transparent;font-variant-numeric:normal;font-variant-east-asian:normal;vertical-align:baseline;white-space:pre-wrap">Network Interfaces</span></p><p dir="ltr" style="line-height:1.2;margin-top:2pt;margin-bottom:2pt"><span style="font-size:10pt;font-family:Arial;color:rgb(0,0,0);background-color:transparent;font-variant-numeric:normal;font-variant-east-asian:normal;vertical-align:baseline;white-space:pre-wrap">(5 e1000e Interfaces including the Management interface)</span></p></td><td style="border-width:0.5pt;border-style:solid;border-color:rgb(0,0,0);vertical-align:top;padding:0pt 5.4pt;overflow:hidden"><p dir="ltr" style="line-height:1.2;margin-top:2pt;margin-bottom:2pt"><span style="font-size:10pt;font-family:Arial;color:rgb(0,0,0);background-color:transparent;font-weight:700;font-variant-numeric:normal;font-variant-east-asian:normal;vertical-align:baseline;white-space:pre-wrap">Network Source</span></p></td><td style="border-width:0.5pt;border-style:solid;border-color:rgb(0,0,0);vertical-align:top;padding:0pt 5.4pt;overflow:hidden"><p dir="ltr" style="line-height:1.2;margin-top:2pt;margin-bottom:2pt"><span style="font-size:10pt;font-family:Arial;color:rgb(0,0,0);background-color:transparent;font-weight:700;font-variant-numeric:normal;font-variant-east-asian:normal;vertical-align:baseline;white-space:pre-wrap">Name</span></p></td><td style="border-width:0.5pt;border-style:solid;border-color:rgb(0,0,0);vertical-align:top;padding:0pt 5.4pt;overflow:hidden"><p dir="ltr" style="line-height:1.2;margin-top:2pt;margin-bottom:2pt"><span style="font-size:10pt;font-family:Arial;color:rgb(0,0,0);background-color:transparent;font-weight:700;font-variant-numeric:normal;font-variant-east-asian:normal;vertical-align:baseline;white-space:pre-wrap">Mode</span></p></td></tr><tr style="height:10pt"><td style="border-width:0.5pt;border-style:solid;border-color:rgb(0,0,0);vertical-align:top;padding:0pt 5.4pt;overflow:hidden"><p dir="ltr" style="line-height:1.2;margin-top:2pt;margin-bottom:2pt"><span style="font-size:10pt;font-family:Arial;color:rgb(0,0,0);background-color:transparent;font-variant-numeric:normal;font-variant-east-asian:normal;vertical-align:baseline;white-space:pre-wrap">Host Device</span></p><p dir="ltr" style="line-height:1.2;margin-top:2pt;margin-bottom:2pt"><span style="font-size:10pt;font-family:Arial;color:rgb(0,0,0);background-color:transparent;font-variant-numeric:normal;font-variant-east-asian:normal;vertical-align:baseline;white-space:pre-wrap">(Management)</span></p></td><td style="border-width:0.5pt;border-style:solid;border-color:rgb(0,0,0);vertical-align:top;padding:0pt 5.4pt;overflow:hidden"><p dir="ltr" style="line-height:1.2;margin-top:2pt;margin-bottom:2pt"><span style="font-size:10pt;font-family:Arial;color:rgb(0,0,0);background-color:transparent;font-variant-numeric:normal;font-variant-east-asian:normal;vertical-align:baseline;white-space:pre-wrap">Interface-1</span></p></td><td style="border-width:0.5pt;border-style:solid;border-color:rgb(0,0,0);vertical-align:top;padding:0pt 5.4pt;overflow:hidden"><p dir="ltr" style="line-height:1.2;margin-top:2pt;margin-bottom:2pt"><span style="font-size:10pt;font-family:Arial;color:rgb(0,0,0);background-color:transparent;font-variant-numeric:normal;font-variant-east-asian:normal;vertical-align:baseline;white-space:pre-wrap">Bridge</span></p></td></tr><tr style="height:10pt"><td rowspan="4" style="border-width:0.5pt;border-style:solid;border-color:rgb(0,0,0);vertical-align:top;padding:0pt 5.4pt;overflow:hidden"><br><br><p dir="ltr" style="line-height:1.2;margin-top:2pt;margin-bottom:2pt"><span style="font-size:10pt;font-family:Arial;color:rgb(0,0,0);background-color:transparent;font-variant-numeric:normal;font-variant-east-asian:normal;vertical-align:baseline;white-space:pre-wrap">Host Device</span></p></td><td style="border-width:0.5pt;border-style:solid;border-color:rgb(0,0,0);vertical-align:top;padding:0pt 5.4pt;overflow:hidden"><p dir="ltr" style="line-height:1.2;margin-top:2pt;margin-bottom:2pt"><span style="font-size:10pt;font-family:Arial;color:rgb(0,0,0);background-color:transparent;font-variant-numeric:normal;font-variant-east-asian:normal;vertical-align:baseline;white-space:pre-wrap">Interface-2</span></p></td><td style="border-width:0.5pt;border-style:solid;border-color:rgb(0,0,0);vertical-align:top;padding:0pt 5.4pt;overflow:hidden"><p dir="ltr" style="line-height:1.2;margin-top:2pt;margin-bottom:2pt"><span style="font-size:10pt;font-family:Arial;color:rgb(0,0,0);background-color:transparent;font-variant-numeric:normal;font-variant-east-asian:normal;vertical-align:baseline;white-space:pre-wrap">Bridge</span></p></td></tr><tr style="height:10pt"><td style="border-width:0.5pt;border-style:solid;border-color:rgb(0,0,0);vertical-align:top;padding:0pt 5.4pt;overflow:hidden"><p dir="ltr" style="line-height:1.2;margin-top:2pt;margin-bottom:2pt"><span style="font-size:10pt;font-family:Arial;color:rgb(0,0,0);background-color:transparent;font-variant-numeric:normal;font-variant-east-asian:normal;vertical-align:baseline;white-space:pre-wrap">Interface-3</span></p></td><td style="border-width:0.5pt;border-style:solid;border-color:rgb(0,0,0);vertical-align:top;padding:0pt 5.4pt;overflow:hidden"><p dir="ltr" style="line-height:1.2;margin-top:2pt;margin-bottom:2pt"><span style="font-size:10pt;font-family:Arial;color:rgb(0,0,0);background-color:transparent;font-variant-numeric:normal;font-variant-east-asian:normal;vertical-align:baseline;white-space:pre-wrap">Bridge</span></p></td></tr><tr style="height:10pt"><td style="border-width:0.5pt;border-style:solid;border-color:rgb(0,0,0);vertical-align:top;padding:0pt 5.4pt;overflow:hidden"><p dir="ltr" style="line-height:1.2;margin-top:2pt;margin-bottom:2pt"><span style="font-size:10pt;font-family:Arial;color:rgb(0,0,0);background-color:transparent;font-variant-numeric:normal;font-variant-east-asian:normal;vertical-align:baseline;white-space:pre-wrap">Interface-4</span></p></td><td style="border-width:0.5pt;border-style:solid;border-color:rgb(0,0,0);vertical-align:top;padding:0pt 5.4pt;overflow:hidden"><p dir="ltr" style="line-height:1.2;margin-top:2pt;margin-bottom:2pt"><span style="font-size:10pt;font-family:Arial;color:rgb(0,0,0);background-color:transparent;font-variant-numeric:normal;font-variant-east-asian:normal;vertical-align:baseline;white-space:pre-wrap">Bridge</span></p></td></tr><tr style="height:10pt"><td style="border-width:0.5pt;border-style:solid;border-color:rgb(0,0,0);vertical-align:top;padding:0pt 5.4pt;overflow:hidden"><p dir="ltr" style="line-height:1.2;margin-top:2pt;margin-bottom:2pt"><span style="font-size:10pt;font-family:Arial;color:rgb(0,0,0);background-color:transparent;font-variant-numeric:normal;font-variant-east-asian:normal;vertical-align:baseline;white-space:pre-wrap">Interface-5</span></p></td><td style="border-width:0.5pt;border-style:solid;border-color:rgb(0,0,0);vertical-align:top;padding:0pt 5.4pt;overflow:hidden"><p dir="ltr" style="line-height:1.2;margin-top:2pt;margin-bottom:2pt"><span style="font-size:10pt;font-family:Arial;color:rgb(0,0,0);background-color:transparent;font-variant-numeric:normal;font-variant-east-asian:normal;vertical-align:baseline;white-space:pre-wrap">Bridge</span></p></td></tr><tr style="height:10pt"><td rowspan="3" style="border-width:0.5pt;border-style:solid;border-color:rgb(0,0,0);vertical-align:top;padding:0pt 5.4pt;overflow:hidden"><br><p dir="ltr" style="line-height:1.2;margin-top:2pt;margin-bottom:2pt"><span style="font-size:10pt;font-family:Arial;color:rgb(0,0,0);background-color:transparent;font-variant-numeric:normal;font-variant-east-asian:normal;vertical-align:baseline;white-space:pre-wrap">Virtual Network(open Network)</span></p></td><td style="border-width:0.5pt;border-style:solid;border-color:rgb(0,0,0);vertical-align:top;padding:0pt 5.4pt;overflow:hidden"><p dir="ltr" style="line-height:1.2;margin-top:2pt;margin-bottom:2pt"><span style="font-size:10pt;font-family:Arial;color:rgb(0,0,0);background-color:transparent;font-variant-numeric:normal;font-variant-east-asian:normal;vertical-align:baseline;white-space:pre-wrap">Interface-6</span></p></td><td style="border-width:0.5pt;border-style:solid;border-color:rgb(0,0,0);vertical-align:top;padding:0pt 5.4pt;overflow:hidden"><p dir="ltr" style="line-height:1.2;margin-top:2pt;margin-bottom:2pt"><span style="font-size:10pt;font-family:Arial;color:rgb(0,0,0);background-color:transparent;font-variant-numeric:normal;font-variant-east-asian:normal;vertical-align:baseline;white-space:pre-wrap">Open</span></p></td></tr><tr style="height:10pt"><td style="border-width:0.5pt;border-style:solid;border-color:rgb(0,0,0);vertical-align:top;padding:0pt 5.4pt;overflow:hidden"><p dir="ltr" style="line-height:1.2;margin-top:2pt;margin-bottom:2pt"><span style="font-size:10pt;font-family:Arial;color:rgb(0,0,0);background-color:transparent;font-variant-numeric:normal;font-variant-east-asian:normal;vertical-align:baseline;white-space:pre-wrap">Interface-7</span></p></td><td style="border-width:0.5pt;border-style:solid;border-color:rgb(0,0,0);vertical-align:top;padding:0pt 5.4pt;overflow:hidden"><p dir="ltr" style="line-height:1.2;margin-top:2pt;margin-bottom:2pt"><span style="font-size:10pt;font-family:Arial;color:rgb(0,0,0);background-color:transparent;font-variant-numeric:normal;font-variant-east-asian:normal;vertical-align:baseline;white-space:pre-wrap">Open</span></p></td></tr><tr style="height:10pt"><td style="border-width:0.5pt;border-style:solid;border-color:rgb(0,0,0);vertical-align:top;padding:0pt 5.4pt;overflow:hidden"><p dir="ltr" style="line-height:1.2;margin-top:2pt;margin-bottom:2pt"><span style="font-size:10pt;font-family:Arial;color:rgb(0,0,0);background-color:transparent;font-variant-numeric:normal;font-variant-east-asian:normal;vertical-align:baseline;white-space:pre-wrap">Interface-8</span></p></td><td style="border-width:0.5pt;border-style:solid;border-color:rgb(0,0,0);vertical-align:top;padding:0pt 5.4pt;overflow:hidden"><p dir="ltr" style="line-height:1.2;margin-top:2pt;margin-bottom:2pt"><span style="font-size:10pt;font-family:Arial;color:rgb(0,0,0);background-color:transparent;font-variant-numeric:normal;font-variant-east-asian:normal;vertical-align:baseline;white-space:pre-wrap">Open</span></p></td></tr></tbody></table></div></span></div><div><br></div><div>Can you please suggest the best recommended drivers and internal network.</div><div><br></div><div>Regards,</div><div>illaya</div></div>