<html xmlns:v="urn:schemas-microsoft-com:vml" xmlns:o="urn:schemas-microsoft-com:office:office" xmlns:w="urn:schemas-microsoft-com:office:word" xmlns:m="http://schemas.microsoft.com/office/2004/12/omml" xmlns="http://www.w3.org/TR/REC-html40"> <head> <meta http-equiv="Content-Type" content="text/html; charset=iso-8859-1"> <meta name="Generator" content="Microsoft Word 15 (filtered medium)"> <style></style> </head> <body lang="EN-US" link="blue" vlink="purple" style="word-wrap:break-word"> <div class="WordSection1"> Hi Morten,<o:p></o:p> <o:p> </o:p> That's *exactly* how this architecture is designed & implemented.<o:p></o:p> <ol style="margin-top:0cm" start="1" type="1"> <li class="MsoListParagraph" style="color:black;margin-left:0cm;mso-list:l1 level1 lfo5"> The DMA configuration and initialization is up to the application (OVS).<o:p></o:p></li><li class="MsoListParagraph" style="color:black;margin-left:0cm;mso-list:l1 level1 lfo5"> The VHost library is passed the DMA-dev ID, and its new async rx/tx APIs, and uses the DMA device to accelerate the copy.<o:p></o:p></li></ol> <o:p> </o:p> Looking forward to talking on the call that just started. Regards, -Harry<o:p></o:p> <o:p> </o:p> <o:p> </o:p> <div style="border:none;border-left:solid blue 1.5pt;padding:0cm 0cm 0cm 4.0pt"> <div> <div style="border:none;border-top:solid #E1E1E1 1.0pt;padding:3.0pt 0cm 0cm 0cm"> From: Morten Br�rup <mb@smartsharesystems.com> Sent: Tuesday, March 29, 2022 1:51 PM To: Pai G, Sunil <sunil.pai.g@intel.com>; Stokes, Ian <ian.stokes@intel.com>; Hu, Jiayu <jiayu.hu@intel.com>; Ferriter, Cian <cian.ferriter@intel.com>; Van Haaren, Harry <harry.van.haaren@intel.com>; Ilya Maximets <i.maximets@ovn.org>; maxime.coquelin@redhat.com; ovs-dev@openvswitch.org; dev@dpdk.org Cc: Mcnamara, John <john.mcnamara@intel.com>; O'Driscoll, Tim <tim.odriscoll@intel.com>; Finn, Emma <emma.finn@intel.com> Subject: RE: OVS DPDK DMA-Dev library/Design Discussion<o:p></o:p> </div> </div> <o:p> </o:p> Having thought more about it, I think that a completely different architectural approach is required:<o:p></o:p> <o:p> </o:p> Many of the DPDK Ethernet PMDs implement a variety of RX and TX packet burst functions, each optimized for different CPU vector instruction sets. The availability of a DMA engine should be treated the same way. So I suggest that PMDs copying packet contents, e.g. memif, pcap, vmxnet3, should implement DMA optimized RX and TX packet burst functions.<o:p></o:p> <o:p> </o:p> Similarly for the DPDK vhost library.<o:p></o:p> <o:p> </o:p> In such an architecture, it would be the application’s job to allocate DMA channels and assign them to the specific PMDs that should use them. But the actual use of the DMA channels would move down below the application and into the DPDK PMDs and libraries.<o:p></o:p> <o:p> </o:p> <o:p> </o:p> <div> Med venlig hilsen / Kind regards,<o:p></o:p> -Morten Br�rup<o:p></o:p> </div> <o:p> </o:p> <div style="border:none;border-left:solid blue 1.5pt;padding:0cm 0cm 0cm 4.0pt"> <div> <div style="border:none;border-top:solid #B5C4DF 1.0pt;padding:3.0pt 0cm 0cm 0cm"> From: Pai G, Sunil [<a href="mailto:sunil.pai.g@intel.com">mailto:sunil.pai.g@intel.com</a>] Sent: Monday, 28 March 2022 20.19 To: Stokes, Ian; Hu, Jiayu; Ferriter, Cian; Van Haaren, Harry; Ilya Maximets; Maxime Coquelin (<a href="mailto:maxime.coquelin@redhat.com">maxime.coquelin@redhat.com</a>); <a href="mailto:ovs-dev@openvswitch.org">ovs-dev@openvswitch.org</a>; <a href="mailto:dev@dpdk.org"> dev@dpdk.org</a> Cc: Mcnamara, John; O'Driscoll, Tim; Finn, Emma Subject: RE: OVS DPDK DMA-Dev library/Design Discussion<o:p></o:p> </div> </div> <o:p> </o:p> <div> Hi All, <o:p></o:p> </div> <div> <o:p></o:p> </div> <div> Please see below PDF which will be presented in the call.<o:p></o:p> </div> <div> <a href="https://github.com/Sunil-Pai-G/OVS-DPDK-presentation-share/blob/main/OVS%20vhost%20async%20datapath%20design%202022%20session%202.pdf">https://github.com/Sunil-Pai-G/OVS-DPDK-presentation-share/blob/main/OVS%20vhost%20async%20datapath%20design%202022%20session%202.pdf</a> <o:p></o:p> </div> <div> <o:p></o:p> </div> <div> Thanks and Regards,<o:p></o:p> </div> <div> Sunil<o:p></o:p> </div> <div> <o:p></o:p> </div> <div> <o:p></o:p> </div> <div> -----Original Appointment----- From: Stokes, Ian <<a href="mailto:ian.stokes@intel.com">ian.stokes@intel.com</a>> Sent: Thursday, March 24, 2022 9:07 PM To: Pai G, Sunil; Hu, Jiayu; Ferriter, Cian; Van Haaren, Harry; Ilya Maximets; Maxime Coquelin (<a href="mailto:maxime.coquelin@redhat.com">maxime.coquelin@redhat.com</a>); <a href="mailto:ovs-dev@openvswitch.org">ovs-dev@openvswitch.org</a>; <a href="mailto:dev@dpdk.org"> dev@dpdk.org</a> Cc: Mcnamara, John; O'Driscoll, Tim; Finn, Emma Subject: OVS DPDK DMA-Dev library/Design Discussion When: Tuesday, March 29, 2022 2:00 PM-3:00 PM (UTC+00:00) Dublin, Edinburgh, Lisbon, London. Where: Google Meet<o:p></o:p> </div> <div> <o:p></o:p> </div> <div> <o:p></o:p> </div> <div> Hi All,<o:p></o:p> </div> <div> <o:p></o:p> </div> <div> This meeting is a follow up to the call earlier this week.<o:p></o:p> </div> <div> <o:p></o:p> </div> <div> This week Sunil presented 3 different approaches to integrating DMA-Dev with OVS along with the performance impacts.<o:p></o:p> </div> <div> <o:p></o:p> </div> <div> <a href="https://github.com/Sunil-Pai-G/OVS-DPDK-presentation-share/blob/main/OVS%20vhost%20async%20datapath%20design%202022.pdf">https://github.com/Sunil-Pai-G/OVS-DPDK-presentation-share/blob/main/OVS%20vhost%20async%20datapath%20design%202022.pdf</a><o:p></o:p> </div> <div> <o:p></o:p> </div> <div> The approaches were as follows:<o:p></o:p> </div> <div> <o:p></o:p> </div> <![if !supportLists]>� <![endif]>Defer work.<o:p></o:p> <![if !supportLists]>� <![endif]>Tx completions from Rx context.<o:p></o:p> <![if !supportLists]>� <![endif]>Tx completions from Rx context + lockless ring.<o:p></o:p> <div> <o:p></o:p> </div> <div> The pros and cons of each approach were discussed but there was no clear solution reached.<o:p></o:p> </div> <div> <o:p></o:p> </div> <div> As such a follow up call was suggested to continue discussion and to reach a clear decision on the approach to take.<o:p></o:p> </div> <div> <o:p></o:p> </div> <div> Please see agenda as it stands below:<o:p></o:p> </div> <div> <o:p></o:p> </div> <div> Agenda<o:p></o:p> </div> <![if !supportLists]>� <![endif]>Opens<o:p></o:p> <![if !supportLists]>� <![endif]>Continue discussion of 3x approaches from last week (Defer work, “V3”, V4, links to patches in Sunil’s slides above)<o:p></o:p> <![if !supportLists]>� <![endif]>Design Feedback (please review solutions of above & slide-deck from last week before call to be informed)<o:p></o:p> <![if !supportLists]>� <![endif]>Dynamic Allocation of DMA engine per queue<o:p></o:p> <![if !supportLists]>� <![endif]>Code Availability (DPDK GitHub, OVS GitHub branches)<o:p></o:p> <div> <o:p></o:p> </div> <div> Please feel free to respond with any other items to be added to the agenda.<o:p></o:p> </div> <div> <o:p></o:p> </div> <div> Google Meet: <a href="https://meet.google.com/hme-pygf-bfb">https://meet.google.com/hme-pygf-bfb</a><o:p></o:p> </div> <div> <o:p></o:p> </div> <div> Regards<o:p></o:p> </div> <div> Ian<o:p></o:p> </div> <div> <o:p></o:p> </div> </div> </div> </div> </body> </html>