<html xmlns:o="urn:schemas-microsoft-com:office:office" xmlns:w="urn:schemas-microsoft-com:office:word" xmlns:m="http://schemas.microsoft.com/office/2004/12/omml" xmlns="http://www.w3.org/TR/REC-html40">
<head>
<meta http-equiv="Content-Type" content="text/html; charset=Windows-1252">
<meta name="Generator" content="Microsoft Word 15 (filtered medium)">
<style><!--
/* Font Definitions */
@font-face
{font-family:"Cambria Math";
panose-1:2 4 5 3 5 4 6 3 2 4;}
@font-face
{font-family:Calibri;
panose-1:2 15 5 2 2 2 4 3 2 4;}
/* Style Definitions */
p.MsoNormal, li.MsoNormal, div.MsoNormal
{margin:0in;
font-size:10.0pt;
font-family:"Calibri",sans-serif;}
.MsoChpDefault
{mso-style-type:export-only;
font-size:10.0pt;
mso-ligatures:none;}
@page WordSection1
{size:8.5in 11.0in;
margin:1.0in 1.0in 1.0in 1.0in;}
div.WordSection1
{page:WordSection1;}
--></style>
</head>
<body lang="EN-US" link="#0563C1" vlink="#954F72" style="word-wrap:break-word">
<div class="WordSection1">
<p class="MsoNormal"><span style="font-size:11.0pt">Hi Maxime,<o:p></o:p></span></p>
<p class="MsoNormal"><span style="font-size:11.0pt">I back ported the patch to v22.11.2 and it worked for us on both the testpmd app and with our 32-bit DPDK (virtio-pci) application. The change to set the mbuf_addr_mask was moved under virtio_init_queue()
in v22.11.2 (see below).<o:p></o:p></span></p>
<p class="MsoNormal"><span style="font-size:11.0pt"><o:p> </o:p></span></p>
<p class="MsoNormal"><span style="font-size:11.0pt">I’m in the process of updating the application to v23.07 and will test there as well. Thank you for looking into this and providing the patch.<o:p></o:p></span></p>
<p class="MsoNormal"><span style="font-size:11.0pt">Regards, Dave<o:p></o:p></span></p>
<p class="MsoNormal" style="margin-left:.5in"><span style="font-size:11.0pt"><o:p> </o:p></span></p>
<p class="MsoNormal" style="margin-left:.5in"><span style="font-size:11.0pt">diff --git a/drivers/net/virtio/virtio_ethdev.c b/drivers/net/virtio/virtio_ethdev.c<o:p></o:p></span></p>
<p class="MsoNormal" style="margin-left:.5in"><span style="font-size:11.0pt">index b72334455e..bd90ba9d49 100644<o:p></o:p></span></p>
<p class="MsoNormal" style="margin-left:.5in"><span style="font-size:11.0pt">--- a/drivers/net/virtio/virtio_ethdev.c<o:p></o:p></span></p>
<p class="MsoNormal" style="margin-left:.5in"><span style="font-size:11.0pt">+++ b/drivers/net/virtio/virtio_ethdev.c<o:p></o:p></span></p>
<p class="MsoNormal" style="margin-left:.5in"><span style="font-size:11.0pt">@@ -565,10 +565,13 @@ virtio_init_queue(struct rte_eth_dev *dev, uint16_t queue_idx)<o:p></o:p></span></p>
<p class="MsoNormal" style="margin-left:.5in"><span style="font-size:11.0pt"><o:p></o:p></span></p>
<p class="MsoNormal" style="margin-left:.5in"><span style="font-size:11.0pt"> memset(mz->addr, 0, mz->len);<o:p></o:p></span></p>
<p class="MsoNormal" style="margin-left:.5in"><span style="font-size:11.0pt"><o:p></o:p></span></p>
<p class="MsoNormal" style="margin-left:.5in"><span style="font-size:11.0pt">- if (hw->use_va)<o:p></o:p></span></p>
<p class="MsoNormal" style="margin-left:.5in"><span style="font-size:11.0pt">+ if (hw->use_va) {<o:p></o:p></span></p>
<p class="MsoNormal" style="margin-left:.5in"><span style="font-size:11.0pt"> vq->vq_ring_mem = (uintptr_t)mz->addr;<o:p></o:p></span></p>
<p class="MsoNormal" style="margin-left:.5in"><span style="font-size:11.0pt">- else<o:p></o:p></span></p>
<p class="MsoNormal" style="margin-left:.5in"><span style="font-size:11.0pt">+ vq->mbuf_addr_mask = UINTPTR_MAX;<o:p></o:p></span></p>
<p class="MsoNormal" style="margin-left:.5in"><span style="font-size:11.0pt">+ } else {<o:p></o:p></span></p>
<p class="MsoNormal" style="margin-left:.5in"><span style="font-size:11.0pt"> vq->vq_ring_mem = mz->iova;<o:p></o:p></span></p>
<p class="MsoNormal" style="margin-left:.5in"><span style="font-size:11.0pt">+ vq->mbuf_addr_mask = UINT64_MAX;<o:p></o:p></span></p>
<p class="MsoNormal" style="margin-left:.5in"><span style="font-size:11.0pt">+ }<o:p></o:p></span></p>
<p class="MsoNormal" style="margin-left:.5in"><span style="font-size:11.0pt"><o:p></o:p></span></p>
<p class="MsoNormal" style="margin-left:.5in"><span style="font-size:11.0pt"> vq->vq_ring_virt_mem = mz->addr;<o:p></o:p></span></p>
<p class="MsoNormal" style="margin-left:.5in"><span style="font-size:11.0pt"> PMD_INIT_LOG(DEBUG, "vq->vq_ring_mem: 0x%" PRIx64, vq->vq_ring_mem);<o:p></o:p></span></p>
<p class="MsoNormal"><span style="font-size:11.0pt"><o:p> </o:p></span></p>
<div id="mail-editor-reference-message-container">
<div>
<div style="border:none;border-top:solid #B5C4DF 1.0pt;padding:3.0pt 0in 0in 0in">
<p class="MsoNormal" style="margin-bottom:12.0pt"><b><span style="font-size:12.0pt;color:black">From:
</span></b><span style="font-size:12.0pt;color:black">Maxime Coquelin <maxime.coquelin@redhat.com><br>
<b>Date: </b>Wednesday, September 20, 2023 at 9:02 AM<br>
<b>To: </b>dev@dpdk.org <dev@dpdk.org>, Roger Melton (rmelton) <rmelton@cisco.com>, Dave Johnson (davejo) <davejo@cisco.com>, Sampath Peechu (speechu) <speechu@cisco.com>, chenbo.xia@outlook.com <chenbo.xia@outlook.com>, Malcolm Bumgardner (mbumgard) <mbumgard@cisco.com>,
Chris Brezovec (cbrezove) <cbrezove@cisco.com>, david.marchand@redhat.com <david.marchand@redhat.com><br>
<b>Cc: </b>Maxime Coquelin <maxime.coquelin@redhat.com>, stable@dpdk.org <stable@dpdk.org><br>
<b>Subject: </b>[PATCH] net/virtio: fix descriptors buffer addresses on 32 bits builds<o:p></o:p></span></p>
</div>
<div>
<p class="MsoNormal" style="margin-bottom:12.0pt"><span style="font-size:11.0pt">With Virtio-user, the Virtio descriptor buffer address is the<br>
virtual address of the mbuf's buffer. On 32 bits builds, it is<br>
expected to be 32 bits.<br>
<br>
With Virtio-PCI, the Virtio descriptor buffer address is the<br>
physical address of the mbuf's buffer. On 32 bits builds running<br>
on 64 bits kernel, it is expected to be up to 64 bits.<br>
<br>
This patch introduces a new mask field in virtqueue's struct to<br>
filter our the upper 4 bytes of the address only when necessary.<br>
An optimization is introduced for 64 bits builds to remove the<br>
masking, as the address is always 64 bits wide.<br>
<br>
Fixes: ba55c94a7ebc ("net/virtio: revert forcing IOVA as VA mode for virtio-user")<br>
Cc: stable@dpdk.org<br>
<br>
Reported-by: Sampath Peechu <speechu@cisco.com><br>
Signed-off-by: Maxime Coquelin <maxime.coquelin@redhat.com><br>
---<br>
drivers/net/virtio/virtqueue.c | 2 ++<br>
drivers/net/virtio/virtqueue.h | 18 ++++++++++++++----<br>
2 files changed, 16 insertions(+), 4 deletions(-)<br>
<br>
diff --git a/drivers/net/virtio/virtqueue.c b/drivers/net/virtio/virtqueue.c<br>
index 1d836f2530..6f419665f1 100644<br>
--- a/drivers/net/virtio/virtqueue.c<br>
+++ b/drivers/net/virtio/virtqueue.c<br>
@@ -469,9 +469,11 @@ virtqueue_alloc(struct virtio_hw *hw, uint16_t index, uint16_t num, int type,<br>
if (hw->use_va) {<br>
vq->vq_ring_mem = (uintptr_t)mz->addr;<br>
vq->mbuf_addr_offset = offsetof(struct rte_mbuf, buf_addr);<br>
+ vq->mbuf_addr_mask = UINTPTR_MAX;<br>
} else {<br>
vq->vq_ring_mem = mz->iova;<br>
vq->mbuf_addr_offset = offsetof(struct rte_mbuf, buf_iova);<br>
+ vq->mbuf_addr_mask = UINT64_MAX;<br>
}<br>
<br>
PMD_INIT_LOG(DEBUG, "vq->vq_ring_mem: 0x%" PRIx64, vq->vq_ring_mem);<br>
diff --git a/drivers/net/virtio/virtqueue.h b/drivers/net/virtio/virtqueue.h<br>
index 9d4aba11a3..c1cb941c43 100644<br>
--- a/drivers/net/virtio/virtqueue.h<br>
+++ b/drivers/net/virtio/virtqueue.h<br>
@@ -114,17 +114,26 @@ virtqueue_store_flags_packed(struct vring_packed_desc *dp,<br>
<br>
#define VIRTQUEUE_MAX_NAME_SZ 32<br>
<br>
+#ifdef RTE_ARCH_32<br>
+#define VIRTIO_MBUF_ADDR_MASK(vq) ((vq)->mbuf_addr_mask)<br>
+#else<br>
+#define VIRTIO_MBUF_ADDR_MASK(vq) UINT64_MAX<br>
+#endif<br>
+<br>
/**<br>
* Return the IOVA (or virtual address in case of virtio-user) of mbuf<br>
* data buffer.<br>
*<br>
* The address is firstly casted to the word size (sizeof(uintptr_t))<br>
- * before casting it to uint64_t. This is to make it work with different<br>
- * combination of word size (64 bit and 32 bit) and virtio device<br>
- * (virtio-pci and virtio-user).<br>
+ * before casting it to uint64_t. It is then masked with the expected<br>
+ * address length (64 bits for virtio-pci, word size for virtio-user).<br>
+ *<br>
+ * This is to make it work with different combination of word size (64<br>
+ * bit and 32 bit) and virtio device (virtio-pci and virtio-user).<br>
*/<br>
#define VIRTIO_MBUF_ADDR(mb, vq) \<br>
- ((uint64_t)(*(uintptr_t *)((uintptr_t)(mb) + (vq)->mbuf_addr_offset)))<br>
+ ((*(uint64_t *)((uintptr_t)(mb) + (vq)->mbuf_addr_offset)) & \<br>
+ VIRTIO_MBUF_ADDR_MASK(vq))<br>
<br>
/**<br>
* Return the physical address (or virtual address in case of<br>
@@ -194,6 +203,7 @@ struct virtqueue {<br>
void *vq_ring_virt_mem; /**< linear address of vring*/<br>
unsigned int vq_ring_size;<br>
uint16_t mbuf_addr_offset;<br>
+ uint64_t mbuf_addr_mask;<br>
<br>
union {<br>
struct virtnet_rx rxq;<br>
-- <br>
2.41.0<o:p></o:p></span></p>
</div>
</div>
</div>
</div>
</body>
</html>