[dpdk-dev] [PATCH] igb_uio: map dummy dma forcing iommu domain attachment

[Alejandro Lucero]

On Fri, Feb 10, 2017 at 7:06 PM, Ferruh Yigit <ferruh.yigit at intel.com>
wrote:

> On 2/10/2017 7:03 PM, Ferruh Yigit wrote:
> > On 2/8/2017 11:54 AM, Alejandro Lucero wrote:
> >> Hi Ferruh,
> >>
> >> On Tue, Feb 7, 2017 at 3:59 PM, Ferruh Yigit <ferruh.yigit at intel.com
> >> <mailto:ferruh.yigit at intel.com>> wrote:
> >>
> >>     Hi Alejandro,
> >>
> >>     On 1/18/2017 12:27 PM, Alejandro Lucero wrote:
> >>     > For using a DPDK app when iommu is enabled, it requires to
> >>     > add iommu=pt to the kernel command line. But using igb_uio driver
> >>     > makes DMAR errors because the device has not an IOMMU domain.
> >>
> >>     Please help to understand the scope of the problem,
> >>
> >>
> >> After reading your reply, I realize I could have explained it better.
> >> First of all, this is related to SRIOV, exactly when the VFs are
> created.
> >>
> >>
> >>     1- How can you re-produce the problem?
> >>
> >>
> >> Using a VF from a Intel card by a DPDK app in the host and a kernel >=
> >> 3.15. Although usually VFs are assigned to VMs, it could also be an
> >> option to use VFs by the host.
> >>
> >> BTW, I did not try to reproduce the problem with an Intel card. I
> >> triggered this problem with an NFP, but because the problem behind, I
> >> bet that is going to happen for an Intel one as well.
> >
> > I can able to reproduce the problem with ixgbe, by using VF on the host.
> >
> > And I verified your patch fixes it, it cause device attached to a vfio
> > group.
>
> I want to send this in a separate mail, since not directly related to
> your patch, but while I was testing with vfio-pci I get lower numbers
> comparing to the igb_uio, which is unexpected AFAIK.
>
> Most probably I am doing something wrong, but I would like to ask if are
> you observing same behavior?
>

Can you tell me which test are you running?

Although both, igb_uio and vfio, allow to work with IOMMU, the first one
requires iommu=pt. It implies a single IOMMU domain already created by the
system with the 1:1 mapping being used.  With VFIO, a specific per device
IOMMU domain is created. Depending on how are you measuring performance,
that specific IOMMU domain creation by the DPDK app could have an impact,
but I do not think that should be really significant. But with IOMMU you
have the same problem than with MMU, there is a IOTLB for IOMMU as there is
a TLB for MMU. Depending on the app, some IOMMU/IOTLB contention is likely.
I have done some experiments and still investigating this during my spare
time. It would be worth a talk about this in the next DPDK meeting.


>
> Thanks,
> ferruh
>
>