[dpdk-dev] Query : Does Binding with vfio-pci is supported inside qemu-kvm guest/vm instance.?

[John Sucaet]

Hi Anatoly,

As you said: There's no reason to use igb_uio, ever!
I would like to ask whether vfio-pci with or without vIOMMU should/could 
work for virtio-pci net devices in the case of a 32-bit dpdk 
application, on a 64-bit kernel (4.9) inside a guest VM 
(qemu-2.10.2-1.fc27)?

I tried both a 64-bit and a 32-bit version of the same application, but 
only in the case of the 64-bit application, the port was found by eal. 
The 32-bit application gave errors like:

EAL: pci_map_resource(): cannot mmap(16, 0xf4a01000, 0x4000, 0x0): 
Invalid argument (0xffffffff)
EAL: Failed to map pci BAR4
EAL:   0000:00:02.0 mapping BAR4 failed: Invalid argument
EAL: Can't write to PCI bar (0) : offset (12)
EAL: Can't read from PCI bar (0) : offset (12)
EAL: Can't read from PCI bar (0) : offset (12)
EAL: Can't write to PCI bar (0) : offset (12)
EAL: Can't read from PCI bar (0) : offset (12)
EAL: Can't write to PCI bar (0) : offset (12)
EAL: Can't read from PCI bar (0) : offset (0)
EAL: Can't write to PCI bar (0) : offset (4)
EAL: Can't write to PCI bar (0) : offset (14)
EAL: Can't read from PCI bar (0) : offset (14)
EAL: Can't read from PCI bar (0) : offset (1a)
EAL: Can't read from PCI bar (0) : offset (1c)
EAL: Can't write to PCI bar (0) : offset (e)
EAL: Can't read from PCI bar (0) : offset (c)
virtio_init_queue(): virtqueue size is not powerof 2
EAL: Requested device 0000:00:02.0 cannot be used

Maybe you have an idea what went wrong here?

By preference, I would like to continue to use the 32-bit application 
which worked fine with the igb_uio driver.


Thanks

John


On 03/12/2019 11:57 AM, Burakov, Anatoly wrote:
> On 12-Mar-19 10:20 AM, Bruce Richardson wrote:
>> On Tue, Mar 12, 2019 at 05:54:39PM +0800, Jason Wang wrote:
>>>
>>> On 2019/3/12 下午5:42, Thanneeru Srinivasulu wrote:
>>>> Thanks Bruce..
>>>>
>>>> On Tue, Mar 12, 2019 at 3:08 PM Bruce Richardson
>>>> <bruce.richardson at intel.com> wrote:
>>>>> On Tue, Mar 12, 2019 at 10:57:55AM +0530, Thanneeru Srinivasulu 
>>>>> wrote:
>>>>>> Hi Everyone.
>>>>>>
>>>>>> I did attached pice  to Guest VM using vfio-pci with qemu 
>>>>>> command, and then
>>>>>> tried binding the pcie bdf with vfio-pci, observing binding 
>>>>>> failure with
>>>>>> vfio-pci.
>>>>>>
>>>>>> Where as when tryied with igb_uio, everything works fine.
>>>>>>
>>>>>> Does Binding with vfio-pci is supported inside VM/guest?
>>>>>>
>>>>> vfio support requires the presence of an IOMMU, and you generally 
>>>>> don't
>>>>> have an IOMMU available in a VM.
>>>>>
>>>>> /Bruce
>>>
>>>
>>> Actually, Qemu support vIOMMU + VFIO in guest[1], all you need is to 
>>> add a
>>> intel IOMMU and enabling caching mode.
>>>
>>> Thanks
>>>
>>>
>>> [1]
>>>
>>> https://www.lfasiallc.com/wp-content/uploads/2017/11/Device-Assignment-with-Nested-Guests-and-DPDK_Peter-Xu.pdf 
>>>
>>>
>> Thanks for the info.
>>
>> /Bruce
>>
>
> One more thing: even without vIOMMU, VFIO has no-IOMMU mode which can 
> be enabled (for a recent-enough kernel). This will make VFIO work even 
> in cases where the guest doesn't have IOMMU emulation. See? There's no 
> reason to use igb_uio, ever! :D
>