[dpdk-dev] Having troubles binding an SR-IOV VF to uio_pci_generic on Amazon instance

[Gleb Natapov]

On Wed, Sep 30, 2015 at 09:50:08PM +0300, Michael S. Tsirkin wrote:
> On Wed, Sep 30, 2015 at 10:43:04AM -0700, Stephen Hemminger wrote:
> > On Wed, 30 Sep 2015 20:39:43 +0300
> > "Michael S. Tsirkin" <mst at redhat.com> wrote:
> > 
> > > On Wed, Sep 30, 2015 at 10:28:07AM -0700, Stephen Hemminger wrote:
> > > > On Wed, 30 Sep 2015 13:37:22 +0300
> > > > Vlad Zolotarov <vladz at cloudius-systems.com> wrote:
> > > > 
> > > > > 
> > > > > 
> > > > > On 09/30/15 00:49, Michael S. Tsirkin wrote:
> > > > > > On Tue, Sep 29, 2015 at 02:46:16PM -0700, Stephen Hemminger wrote:
> > > > > >> On Tue, 29 Sep 2015 23:54:54 +0300
> > > > > >> "Michael S. Tsirkin" <mst at redhat.com> wrote:
> > > > > >>
> > > > > >>> On Tue, Sep 29, 2015 at 07:41:09PM +0300, Vlad Zolotarov wrote:
> > > > > >>>> The security breach motivation u brought in "[RFC PATCH] uio:
> > > > > >>>> uio_pci_generic: Add support for MSI interrupts" thread seems a bit weak
> > > > > >>>> since one u let the userland access to the bar it may do any funny thing
> > > > > >>>> using the DMA engine of the device. This kind of stuff should be prevented
> > > > > >>>> using the iommu and if it's enabled then any funny tricks using MSI/MSI-X
> > > > > >>>> configuration will be prevented too.
> > > > > >>>>
> > > > > >>>> I'm about to send the patch to main Linux mailing list. Let's continue this
> > > > > >>>> discussion there.
> > > > > >>>>    
> > > > > >>> Basically UIO shouldn't be used with devices capable of DMA.
> > > > > >>> Use VFIO for that (yes, this implies an emulated or PV IOMMU).
> > > > > 
> > > > > If there is an IOMMU in the picture there shouldn't be any problem to 
> > > > > use UIO with DMA capable devices.
> > > > > 
> > > > > >>> I don't think this can change.
> > > > > >> Given there is no PV IOMMU and even if there was it would be too slow for DPDK
> > > > > >> use, I can't accept that.
> > > > > > QEMU does allow emulating an iommu.
> > > > > 
> > > > > Amazon's EC2 xen HV doesn't. At least today. Therefore VFIO is not an 
> > > > > option there. And again, it's a general issue not DPDK specific.
> > > > > Today one has to develop some proprietary modules (like igb_uio) to 
> > > > > workaround the issue and this is lame. IMHO uio_pci_generic should
> > > > > be fixed to be able to properly work within any virtualized environment 
> > > > > and not only with KVM.
> > > > > 
> > > > 
> > > > Also VMware (bigger problem) has no IOMMU emulation.
> > > > Other environments as well (Windriver, GCE) have noe IOMMU.
> > > 
> > > Because the use-case of userspace drivers is not important enough?
> > > Without an IOMMU, there's no way to have secure userspace drivers.
> > 
> > Look at Cloudius, there is no necessity of security in guest.
> 
> It's an interesting concept, isn't it?
> 
It is.

> So why not do what Cloudius does, and run this task code in ring 0 then,
> allocating all memory in the kernel range?
> 
Except this is not what Cloudius does. The idea of OSv is that it can
run your regular userspace application, but remove unneeded level of
indirection by bypassing userspace/kernelspace communication (among
other things).  Application still uses virtual, not directly mapped
physical memory like Linux ring 0 has.

You can achieve most of the benefits of kernel bypass on Linux too, but
unlike OSv you need to code for it. UIO is one of those things that
allows that.

> You are increasing interrupt latency by a huge factor by channeling
> interrupts through a scheduler.  Let user install an
> interrupt handler function, and be done with it.
> 
Interrupt latency is not always hugely important. If you enter interrupt
mode only when idle hundred more us on a first packet will not kill you. If
interrupt latency is important then uio may be not the right solution,
but then neither is vfio.

--
			Gleb.