[PATCH v1] bus/pci: revise support PASID control

[Chen, Mike Ximing]

> -----Original Message-----
> From: Chenbo Xia <chenbox at nvidia.com>
> Sent: Tuesday, November 14, 2023 8:54 PM
> To: Sevincer, Abdullah <abdullah.sevincer at intel.com>
> Cc: dev at dpdk.org; jerinj at marvell.com; Chen, Mike Ximing
> <mike.ximing.chen at intel.com>; Richardson, Bruce
> <bruce.richardson at intel.com>; NBU-Contact-Thomas Monjalon (EXTERNAL)
> <thomas at monjalon.net>; Marchand, David <david.marchand at redhat.com>;
> nipun.gupta at amd.com
> Subject: Re: [PATCH v1] bus/pci: revise support PASID control
> 
> On Nov 15, 2023, at 01:39, Sevincer, Abdullah <abdullah.sevincer at intel.com>
> wrote:
> >
> > External email: Use caution opening links or attachments
> >
> >
> >> +I don’t know about the details, so it means for different devices that support
> PASID, they have different offsets?
> >
> >> +Btw, Is this cap still not exposed to user space in latest kernel?
> >
> > Yes, may be different offsets for different devices.
> 
> But why? It’s not standard capability? In my understanding, standard cap should
> have the same offset definitions for all devices.

PASID is a part of extended capabilities. Its offset can be different for different devices.

> 
> > As of now it is not exposed to user. Bruce's test was on 6.2 generic
> > kernel (6.2.0-36-generic)
> 
> Will kernel plan to support that? I can see the related work was done by Intel but
> somehow it’s not merged into kernel. Could you give more information on this?
> 
Hi Chenbo,
As you may know there has been a lot of changes in iommu/vfio/SVA/pasid/SIOV 
support in Linux kernel recently. The PASID used to be disabled, but starting with
kernel 6.2 it is enabled in vfio-pci driver by default.  We did contact the kernel developers
on this issue. They seem to insist that enabling PASID is needed for whatever new features
they are developing. This breaks the DLB PF PMD as DLB HW requires the PASID to be
disable for PF to operate properly (otherwise the HW put DLB in a different mode). We 
will continue to talk to the kernel developers on this issue, but in the meantime would like
to provide this patch so that DPDK PF PMD can still work with latest kernels.

In term of exposing the PASID capability to the user space. We are aware of some patches
Submitted in conjunction to the changes mentioned above, for example, 
https://lkml.iu.edu/hypermail/linux/kernel/2309.3/02380.html
But we don’t know when and if it will be accepted into the kernel. Hopefully the patch will
be accepted so we don’t have to use the hard coded offset.

> If kernel does not want this to be exposed, it means userspace should not access
> this. No?
> 
The action (disabling PASID) only applies the targeted device. In the DLB PF PMD case,
the DPDK has full control of the device via vfio-pci.  It does not affect kernel and any
other device's operation.

Thanks
Mike
> /Chenbo