[dpdk-dev] [PATCH] vhost: Clean iotlb cache on vring stop

[Xia, Chenbo]

Hi Eugenio,

> -----Original Message-----
> From: Eugenio Pérez <eperezma at redhat.com>
> Sent: Saturday, August 28, 2021 12:13 AM
> To: Maxime Coquelin <maxime.coquelin at redhat.com>; Xia, Chenbo
> <chenbo.xia at intel.com>
> Cc: dev at dpdk.org; Pei Zhang <pezhang at redhat.com>; Jason Wang
> <jasowang at redhat.com>
> Subject: [PATCH] vhost: Clean iotlb cache on vring stop

Clean -> clean

> 
> Old IOVA cache entries are left when there is a change on virtio driver
> in VM. In case that all these old entries have iova addresses lesser
> than new iova entries, vhost code will need to iterate all the cache to
> find the new ones. In case of just a new iova entry needed for the new
> translations, this condition will last forever.
> 
> This has been observed in virtio-net to testpmd's vfio-pci driver
> transition, reducing the performance from more than 10Mpps to less than
> 0.07Mpps if the hugepage address was higher than the networking
> buffers. Since all new buffers are contained in this new gigantic page,
> vhost needs to scan IOTLB_CACHE_SIZE - 1 for each translation at worst.

I'm curious why QEMU will not invalidate iotlb when virtio-net driver is removed
(dma region should be unmapped).

And since the perf drop is huge, why not cc to stable and add fix tag?

Thanks,
Chenbo

> 
> Signed-off-by: Eugenio Pérez <eperezma at redhat.com>
> Reported-by: Pei Zhang <pezhang at redhat.com>
> ---
>  lib/vhost/vhost_user.c | 2 ++
>  1 file changed, 2 insertions(+)
> 
> diff --git a/lib/vhost/vhost_user.c b/lib/vhost/vhost_user.c
> index 29a4c9af60..7de48f5333 100644
> --- a/lib/vhost/vhost_user.c
> +++ b/lib/vhost/vhost_user.c
> @@ -2113,6 +2113,8 @@ vhost_user_get_vring_base(struct virtio_net **pdev,
>  	msg->size = sizeof(msg->payload.state);
>  	msg->fd_num = 0;
> 
> +	vhost_user_iotlb_flush_all(vq);
> +
>  	vring_invalidate(dev, vq);
> 
>  	return RTE_VHOST_MSG_RESULT_REPLY;
> --
> 2.27.0