<html xmlns:o="urn:schemas-microsoft-com:office:office" xmlns:w="urn:schemas-microsoft-com:office:word" xmlns:dt="uuid:C2F41010-65B3-11d1-A29F-00AA00C14882" xmlns:m="http://schemas.microsoft.com/office/2004/12/omml" xmlns="http://www.w3.org/TR/REC-html40"> <head> <meta http-equiv="Content-Type" content="text/html; charset=Windows-1252"> <meta name="Generator" content="Microsoft Word 15 (filtered medium)"> <style></style> </head> <body lang="EN-US" link="blue" vlink="purple" style="word-wrap:break-word"> <div class="WordSection1"> <div id="mail-editor-reference-message-container"> <div> Hi Maxime, This email is regarding the following commit: <a href="https://github.com/DPDK/dpdk/commit/ba55c94a7ebc386d2288d6578ed57aad6cb92657">https://github.com/DPDK/dpdk/commit/ba55c94a7ebc386d2288d6578ed57aad6cb92657</a> <o:p> </o:p> A query had been sent previously on this topic (see below) indicating this commit appears to have broken the 32-bit testpmd app and impacted one of our products that runs as a 32-bit DPDK application. We consequently backed the commit out of our product but would prefer to get a fix for it. In the earlier exchange, you had asked if we were using virtio-pci or virtio-user (we are using virtio-pci) and asked for logs which Sampath provided. It’s been a while, so let me now if you need me to send resend those logs or need any other information. <o:p> </o:p> FWIW, I reproduced this using testpmd and noticed that this part of the change seems to be the interesting part (in drivers/net/virtio/virtqueue.h): <o:p> </o:p> /**<o:p></o:p> * Return the IOVA (or virtual address in case of virtio-user) of mbuf<o:p></o:p> * data buffer.<o:p></o:p> *<o:p></o:p> * The address is firstly casted to the word size (sizeof(uintptr_t))<o:p></o:p> * before casting it to uint64_t. This is to make it work with different<o:p></o:p> * combination of word size (64 bit and 32 bit) and virtio device<o:p></o:p> * (virtio-pci and virtio-user).<o:p></o:p> */<o:p></o:p> #define VIRTIO_MBUF_ADDR(mb, vq) \<o:p></o:p> ((uint64_t)(*(uintptr_t *)((uintptr_t)(mb) + (vq)->mbuf_addr_offset))<o:p></o:p> <o:p> </o:p> If I revert just this part of the changeset (by re-using the VIRTIO_MBUF_ADDR to return buf_iova which matches what it had used previously), then 32-bit testpmd is able to receive traffic again: <o:p> </o:p> #define VIRTIO_MBUF_ADDR(mb, vq) (mb->buf_iova)<o:p></o:p> <o:p> </o:p> Looking at the address produced by each of these, I see the address is the same except that the casting results in the upper bits getting cleared: <o:p> </o:p> Address from patch (nonworking case) = 0x58e7c900<o:p></o:p> Address using buf_iova (working case) = 0x158e7c900<o:p></o:p> ::<o:p></o:p> Address from patch (nonworking case) = 0x58e7bfc0<o:p></o:p> Address using buf_iova (working case) = 0x158e7bfc0<o:p></o:p> ::<o:p></o:p> Address from patch (nonworking case) = 0x58e7b680<o:p></o:p> Address using buf_iova (working case) = 0x158e7b680<o:p></o:p> ::<o:p></o:p> <o:p> </o:p> Regards, Dave <o:p></o:p> <div id="mail-editor-reference-message-container"> <div> <div style="border:none;border-top:solid #B5C4DF 1.0pt;padding:3.0pt 0in 0in 0in"> From: Sampath Peechu (speechu) <speechu@cisco.com> Date: Monday, January 30, 2023 at 3:29 PM To: Maxime Coquelin <maxime.coquelin@redhat.com>, chenbo.xia@intel.com <chenbo.xia@intel.com>, dev@dpdk.org <dev@dpdk.org> Cc: Roger Melton (rmelton) <rmelton@cisco.com>, Malcolm Bumgardner (mbumgard) <mbumgard@cisco.com> Subject: Re: Commit broke 32-bit testpmd app<o:p></o:p> </div> Hi Maxime,<o:p></o:p> <o:p></o:p> Could you please let us know if you got a chance to look at the debugs logs I provided?<o:p></o:p> <o:p></o:p> Thanks,<o:p></o:p> Sampath<o:p></o:p> <o:p></o:p> <o:p></o:p> <div style="border:none;border-top:solid #B5C4DF 1.0pt;padding:3.0pt 0in 0in 0in"> From: Sampath Peechu (speechu) <speechu@cisco.com> Date: Tuesday, December 6, 2022 at 1:08 PM To: Maxime Coquelin <maxime.coquelin@redhat.com>, chenbo.xia@intel.com <chenbo.xia@intel.com>, dev@dpdk.org <dev@dpdk.org> Cc: Roger Melton (rmelton) <rmelton@cisco.com> Subject: Re: Commit broke 32-bit testpmd app<o:p></o:p> </div> Hi Maxime,<o:p></o:p> <o:p></o:p> Did you get a chance to look into this?<o:p></o:p> <o:p></o:p> Please let me know if you need anything else.<o:p></o:p> <o:p></o:p> Thanks,<o:p></o:p> Sampath<o:p></o:p> <o:p></o:p> <div style="border:none;border-top:solid #B5C4DF 1.0pt;padding:3.0pt 0in 0in 0in"> From: Sampath Peechu (speechu) <speechu@cisco.com> Date: Wednesday, November 23, 2022 at 5:15 PM To: Maxime Coquelin <maxime.coquelin@redhat.com>, chenbo.xia@intel.com <chenbo.xia@intel.com>, dev@dpdk.org <dev@dpdk.org> Cc: Roger Melton (rmelton) <rmelton@cisco.com> Subject: Re: Commit broke 32-bit testpmd app<o:p></o:p> </div> Hi Maxime,<o:p></o:p> <o:p></o:p> I’m attaching the following for reference.<o:p></o:p> <o:p></o:p> <ul style="margin-top:0in" type="disc"> <li class="MsoListParagraph" style="margin-left:0in;mso-list:l1 level1 lfo3">Instructions for Centos8 test setup<o:p></o:p></li><li class="MsoListParagraph" style="margin-left:0in;mso-list:l1 level1 lfo3">Diffs between the working and non-working versions (working version has the problem commit backed out)<o:p></o:p></li><li class="MsoListParagraph" style="margin-left:0in;mso-list:l1 level1 lfo3">Working logs (stats show that ping packets from neighbor VM can be seen with both 64-bit and 32-bit apps)<o:p></o:p></li><li class="MsoListParagraph" style="margin-left:0in;mso-list:l1 level1 lfo3">Non-working logs (stats show that ping packets from neighbor VM are seen with 64-bit app but NOT seen with 32-bit app)<o:p></o:p></li></ul> <o:p></o:p> <o:p></o:p> ============================ <o:p></o:p> <o:p></o:p> $ sudo ./usertools/dpdk-devbind.py --status<o:p></o:p> <o:p></o:p> Network devices using DPDK-compatible driver<o:p></o:p> ============================================<o:p></o:p> 0000:07:00.0 'Virtio network device 1041' drv=igb_uio unused=<o:p></o:p> 0000:08:00.0 'Virtio network device 1041' drv=igb_uio unused=<o:p></o:p> <o:p></o:p> Network devices using kernel driver<o:p></o:p> ===================================<o:p></o:p> 0000:01:00.0 'Virtio network device 1041' if=enp1s0 drv=virtio-pci unused=igb_uio *Active*<o:p></o:p> <o:p></o:p> …<o:p></o:p> ===========================<o:p></o:p> <o:p></o:p> <o:p></o:p> Thanks,<o:p></o:p> Sampath<o:p></o:p> <o:p></o:p> <o:p></o:p> <o:p></o:p> <div style="border:none;border-top:solid #B5C4DF 1.0pt;padding:3.0pt 0in 0in 0in"> From: Maxime Coquelin <maxime.coquelin@redhat.com> Date: Tuesday, November 22, 2022 at 4:24 AM To: Sampath Peechu (speechu) <speechu@cisco.com>, chenbo.xia@intel.com <chenbo.xia@intel.com>, dev@dpdk.org <dev@dpdk.org> Cc: Roger Melton (rmelton) <rmelton@cisco.com> Subject: Re: Commit broke 32-bit testpmd app<o:p></o:p> </div> <div> Hi, In my initial reply (see below), I also asked if you had logs to share. And wondered whether it happens with Virtio PCI or Virtio-user? Regards, Maxime On 11/16/22 00:30, Sampath Peechu (speechu) wrote: > ++ dev@dpdk.org <<a href="mailto:dev@dpdk.org">mailto:dev@dpdk.org</a>> > > *From: *Maxime Coquelin <maxime.coquelin@redhat.com> > *Date: *Tuesday, November 15, 2022 at 3:19 AM > *To: *Sampath Peechu (speechu) <speechu@cisco.com>, chenbo.xia@intel.com > <chenbo.xia@intel.com> > *Cc: *Roger Melton (rmelton) <rmelton@cisco.com> > *Subject: *Re: Commit broke 32-bit testpmd app > > Hi Sampath, > > > Please add dev@dpdk.org, the upstream mailing list, if this is related > to the upstream DPDK project.If it is using RHEL DPDK package, please > use the appropriate support channels. > > On 11/14/22 23:55, Sampath Peechu (speechu) wrote: > > Hi Virtio Maintainers team, > > > > This email is regarding the following commit. > > > > > <a href="https://github.com/DPDK/dpdk/commit/ba55c94a7ebc386d2288d6578ed57aad6cb92657"> https://github.com/DPDK/dpdk/commit/ba55c94a7ebc386d2288d6578ed57aad6cb92657</a> <<a href="https://github.com/DPDK/dpdk/commit/ba55c94a7ebc386d2288d6578ed57aad6cb92657">https://github.com/DPDK/dpdk/commit/ba55c94a7ebc386d2288d6578ed57aad6cb92657</a>> <https://github.com/DPDK/dpdk/commit/ba55c94a7ebc386d2288d6578ed57aad6cb92657 <<a href="https://github.com/DPDK/dpdk/commit/ba55c94a7ebc386d2288d6578ed57aad6cb92657">https://github.com/DPDK/dpdk/commit/ba55c94a7ebc386d2288d6578ed57aad6cb92657</a>>> > > > > The above commit appears to have broken the 32-bit testpmd app (and > > consequently impacted one of our products that runs as a 32-bit DPDK > > app). The 64-bit testpmd app does not appear to be impacted though. > > We'll need some logs to understand what is going on. > Does it happen with virtio-pci or virtio-user? > > Regards, > Maxime > > > With the commit in place, we didn’t see any packets going through at > > all. After backing out the commit and rebuilding the 32-bit testpmd app > > in our test setup, we were able to pass traffic as expected. > > > > Could you please let us know if this is a known issue? And if there is a > > fix available for it? > > > > Thank you, > > > > Sampath Peechu > > > > Cisco Systems > > > <o:p></o:p> </div> </div> </div> </div> </div> </div> </body> </html>