<html xmlns:v="urn:schemas-microsoft-com:vml" xmlns:o="urn:schemas-microsoft-com:office:office" xmlns:w="urn:schemas-microsoft-com:office:word" xmlns:m="http://schemas.microsoft.com/office/2004/12/omml" xmlns="http://www.w3.org/TR/REC-html40">
<head>
<meta http-equiv="Content-Type" content="text/html; charset=us-ascii">
<meta name="Generator" content="Microsoft Word 15 (filtered medium)">
<style><!--
/* Font Definitions */
@font-face
{font-family:Gautami;
panose-1:2 0 5 0 0 0 0 0 0 0;}
@font-face
{font-family:"Cambria Math";
panose-1:2 4 5 3 5 4 6 3 2 4;}
@font-face
{font-family:Calibri;
panose-1:2 15 5 2 2 2 4 3 2 4;}
@font-face
{font-family:Consolas;
panose-1:2 11 6 9 2 2 4 3 2 4;}
/* Style Definitions */
p.MsoNormal, li.MsoNormal, div.MsoNormal
{margin:0in;
font-size:11.0pt;
font-family:"Calibri",sans-serif;}
a:link, span.MsoHyperlink
{mso-style-priority:99;
color:blue;
text-decoration:underline;}
span.ik-banner
{mso-style-name:ik-banner;
background:#FFF3CD;}
span.ik-footer
{mso-style-name:ik-footer;
background:#FFF3CD;}
span.EmailStyle22
{mso-style-type:personal-reply;
font-family:"Calibri",sans-serif;
color:#1F497D;}
.MsoChpDefault
{mso-style-type:export-only;
font-size:10.0pt;
mso-ligatures:none;}
@page WordSection1
{size:8.5in 11.0in;
margin:1.0in 1.0in 1.0in 1.0in;}
div.WordSection1
{page:WordSection1;}
--></style><!--[if gte mso 9]><xml>
<o:shapedefaults v:ext="edit" spidmax="1026" />
</xml><![endif]--><!--[if gte mso 9]><xml>
<o:shapelayout v:ext="edit">
<o:idmap v:ext="edit" data="1" />
</o:shapelayout></xml><![endif]-->
</head>
<body lang="EN-US" link="blue" vlink="purple" style="word-wrap:break-word">
<div class="WordSection1">
<p class="MsoNormal"><span style="font-size:12.0pt;color:#1F497D">Hi Stephen,<o:p></o:p></span></p>
<p class="MsoNormal"><span style="font-size:12.0pt;color:#1F497D"><o:p> </o:p></span></p>
<p class="MsoNormal"><span style="font-size:12.0pt;color:#1F497D">A parallel question about packet-flow between VFs of the same PF when VFs are assigned to different containers on the same host server<o:p></o:p></span></p>
<p class="MsoNormal"><span style="font-size:12.0pt;color:#1F497D"><o:p></o:p></span></p>
<p class="MsoNormal" style="text-indent:.5in"><span style="font-size:12.0pt;color:#1F497D">Create 2 SRIOV-VFs of a PF in the host and assign them to 2 containers (one VF per container)
<o:p></o:p></span></p>
<p class="MsoNormal" style="text-indent:.5in"><span style="font-size:12.0pt;color:#1F497D">send IP packet from container-1 to container-2 (SRC_MAC address in this ethernet frame = container1 VF’s MAC address, DST_MAC address = container2 VF’s MAC address),<o:p></o:p></span></p>
<p class="MsoNormal" style="margin-left:.5in;text-indent:.5in"><span style="font-size:12.0pt;color:#1F497D">container-1 sends packet by calling
</span><span style="font-size:10.0pt;font-family:Consolas;color:black;background:#D4D4D4">rte_eth_tx_burst()</span><span style="font-size:12.0pt;color:#1F497D"><o:p></o:p></span></p>
<p class="MsoNormal" style="margin-left:.5in;text-indent:.5in"><span style="font-size:12.0pt;color:#1F497D">container-2 is polling for packets from its VF by calling
</span><span style="font-size:10.0pt;font-family:Consolas;color:black;background:#D4D4D4">rte_eth_rx_burst()</span><span style="font-size:12.0pt;color:#1F497D"><o:p></o:p></span></p>
<p class="MsoNormal"><span style="font-size:12.0pt;color:#1F497D"><o:p> </o:p></span></p>
<p class="MsoNormal"><span style="font-size:12.0pt;color:#1F497D">Will the packet in above scenario leave the host server, go the switch and then come back to the same host machine for entering container-2 ?<o:p></o:p></span></p>
<p class="MsoNormal"><span style="font-size:12.0pt;color:#1F497D">Or, is the SRIOV in PF-NIC smart to identify that SRC_MAC and DST_MAC of the ethernet frame are its own VFs and hence it routes the packet locally within the NIC (packet doesn’t reach the switch
at all) ?<o:p></o:p></span></p>
<p class="MsoNormal"><span style="font-size:12.0pt;color:#1F497D"><o:p> </o:p></span></p>
<p class="MsoNormal"><span style="font-size:12.0pt;color:#1F497D">Regards,<o:p></o:p></span></p>
<p class="MsoNormal"><span style="font-size:12.0pt;color:#1F497D">Purnima<o:p></o:p></span></p>
<p class="MsoNormal"><span style="font-size:12.0pt;color:#1F497D"><o:p> </o:p></span></p>
<p class="MsoNormal"><span style="font-size:12.0pt;color:#1F497D"><o:p> </o:p></span></p>
<div>
<div style="border:none;border-top:solid #E1E1E1 1.0pt;padding:3.0pt 0in 0in 0in">
<p class="MsoNormal"><b>From:</b> Stephen Hemminger <stephen@networkplumber.org> <br>
<b>Sent:</b> Wednesday, November 20, 2024 3:34 AM<br>
<b>To:</b> Thea Corinne Rossman <thea.rossman@cs.stanford.edu><br>
<b>Cc:</b> users@dpdk.org<br>
<b>Subject:</b> Re: Containernet (Docker/Container Networking) with DPDK?<o:p></o:p></p>
</div>
</div>
<p class="MsoNormal"><o:p> </o:p></p>
<div>
<div>
<p class="MsoNormal" style="mso-line-height-alt:.75pt;background:white"></p></div><p class="MsoNormal"><!--removed inky banner--><o:p> </o:p></p>
</div>
<p style="margin:0in">On Tue, 19 Nov 2024 13:39:38 -0800<o:p></o:p></p>
<p style="margin:0in">Thea Corinne Rossman <<a href="mailto:thea.rossman@cs.stanford.edu">thea.rossman@cs.stanford.edu</a>> wrote:<o:p></o:p></p>
<p style="margin:0in"> <o:p></o:p></p>
<p style="margin:0in">> This is SO helpful -- thank you so much.<o:p></o:p></p>
<p style="margin:0in">> <o:p></o:p></p>
<p style="margin:0in">> One follow-up question regarding NICs: can multiple containers on the same<o:p></o:p></p>
<p style="margin:0in">> host share the same PCI device? If I have a host NIC with (say) VFIO driver<o:p></o:p></p>
<p style="margin:0in">> binding, do I have to split it with some kind of SR-IOV so that each<o:p></o:p></p>
<p style="margin:0in">> container has its own "NIC" binding? Or, when running DPDK's "devbind"<o:p></o:p></p>
<p style="margin:0in">> script, can I set up each one with the same PCI address?<o:p></o:p></p>
<p style="margin:0in"> <o:p></o:p></p>
<p style="margin:0in"> <o:p></o:p></p>
<p style="margin:0in">Totally depends on what container system you are using.<o:p></o:p></p>
<p style="margin:0in">If you have two containers sharing same exact PCI device, chaos would ensue.<o:p></o:p></p>
<p style="margin:0in">You might be able to make two VF's on host and pass one to each container;<o:p></o:p></p>
<p style="margin:0in">that would make more sense. <o:p></o:p></p>
</div>
</body>
</html>