[dpdk-dev] cannot use virtio-user pmd in IOVA=PA mode anymore

[Maxime Coquelin]

Hi Olivier,

On 9/24/21 10:21, Olivier Matz wrote:
> Hello,
> 
> I recently tested a use-case with our application using the main
> branch of dpdk.org.
> 
> - the application runs inside a standard x86 VM (no IOMMU)
> - there are emulated physical NICs inside the VM
> - we use virtio-user pmds connected to tap interfaces through the
>    vhost-net backend for exception path
> 
> This use-case works with the the stable 20.11 branch, but not with the
> current main dpdk.org branch. The virtio-user driver refuses to start in
> IOVA=PA mode:
> 
>    vdev_probe_all_drivers(): virtio_user0 requires VA IOVA mode but current mode is PA, not initializing
>    EAL: Driver cannot attach the device (virtio_user0)
> 
> This is likely due to these commits:
> 
>    8d935fff5546 ("bus/vdev: add driver IOVA VA mode requirement")
>    17043a2909bb ("net/virtio: force IOVA as VA mode for virtio-user")
> 
> We didn't see this problem before because we are only testing dpdk.org
> main branch on physical machines that have an IOMMU.
> 
> I'm not sure to understand the reasons for which the ability to run a
> virtio-user pmd in IOVA=PA mode was removed.
> 
> The commitlog of 17043a2909bb says:
> 
>    At least Vhost-user backend of Virtio-user PMD requires
>    IOVA as VA mode. Until now, it was implemented as a hack
>    by forcing to use mbuf's buf_addr field instead of buf_iova.
> 
> I don't get why vhost-user backend would require IOVA=VA. Note that we
> also have a use-case where a virtio-user pmd is connected to our
> pmd-vhost in another application, which was working with 20.11.
> 
> If there is a constraint with vhost-user backend, what about vhost-net
> backend?
> 
> Would it make sense to re-allow this feature by somehow reverting
> 17043a2909bb with some additional cleanup?

I think so.

> I am aware that a solution can be to configure qemu to enable a vIOMMU,
> but it is not my preferred solution yet, as it would impact our users
> that do not do this currently.

The initial purpose of this change was to simplify the code, I did not 
thought about this adverse side-effect.

While moving to VFIO with IOMMU enabled would be a wise decision :), I
will work on reverting that patch to restore the old behavior as that
was not something that was agreed when the patch was submitted.

Thanks for reporting the issue,
Maxime

> Thanks!
> Olivier
>