[dpdk-dev] [PATCH v7 3/9] linuxapp/eal_pci: get iommu class

santosh santosh.shukla at caviumnetworks.com
Tue Sep 5 10:59:34 CEST 2017


Hi Anatoly,


On Tuesday 05 September 2017 02:25 PM, Burakov, Anatoly wrote:
>> From: santosh [mailto:santosh.shukla at caviumnetworks.com]
>> Sent: Tuesday, September 5, 2017 9:48 AM
>> To: Burakov, Anatoly <anatoly.burakov at intel.com>; dev at dpdk.org
>> Cc: thomas at monjalon.net; jerin.jacob at caviumnetworks.com;
>> hemant.agrawal at nxp.com; olivier.matz at 6wind.com;
>> maxime.coquelin at redhat.com; Gonzalez Monroy, Sergio
>> <sergio.gonzalez.monroy at intel.com>; Richardson, Bruce
>> <bruce.richardson at intel.com>; shreyansh.jain at nxp.com;
>> gaetan.rivet at 6wind.com; stephen at networkplumber.org;
>> aconole at redhat.com
>> Subject: Re: [PATCH v7 3/9] linuxapp/eal_pci: get iommu class
>>
>> Hi Anatoly,
>>
>>
>> On Monday 04 September 2017 08:38 PM, Burakov, Anatoly wrote:
>>>> From: Santosh Shukla [mailto:santosh.shukla at caviumnetworks.com]
>>>> Sent: Thursday, August 31, 2017 4:26 AM
>>>> To: dev at dpdk.org
>>>> Cc: thomas at monjalon.net; jerin.jacob at caviumnetworks.com;
>>>> hemant.agrawal at nxp.com; olivier.matz at 6wind.com;
>>>> maxime.coquelin at redhat.com; Gonzalez Monroy, Sergio
>>>> <sergio.gonzalez.monroy at intel.com>; Richardson, Bruce
>>>> <bruce.richardson at intel.com>; shreyansh.jain at nxp.com;
>>>> gaetan.rivet at 6wind.com; Burakov, Anatoly
>> <anatoly.burakov at intel.com>;
>>>> stephen at networkplumber.org; aconole at redhat.com; Santosh Shukla
>>>> <santosh.shukla at caviumnetworks.com>
>>>> Subject: [PATCH v7 3/9] linuxapp/eal_pci: get iommu class
>>>>
>>>> Get iommu class of PCI device on the bus and returns preferred iova
>>>> mapping mode for that bus.
>>>>
>>>> Patch also introduces RTE_PCI_DRV_IOVA_AS_VA drv flag.
>>>> Flag used when driver needs to operate in iova=va mode.
>>>>
>>>> Algorithm for iova scheme selection for PCI bus:
>>>> 0. If no device bound then return with RTE_IOVA_DC mapping mode, else
>>>> goto 1).
>>>> 1. Look for device attached to vfio kdrv and has .drv_flag set to
>>>> RTE_PCI_DRV_IOVA_AS_VA.
>>>> 2. Look for any device attached to UIO class of driver.
>>>> 3. Check for vfio-noiommu mode enabled.
>>>>
>>>> If 2) & 3) is false and 1) is true then select mapping scheme as
>> RTE_IOVA_VA.
>>>> Otherwise use default mapping scheme (RTE_IOVA_PA).
>>>>
>>>> Signed-off-by: Santosh Shukla <santosh.shukla at caviumnetworks.com>
>>>> Signed-off-by: Jerin Jacob <jerin.jacob at caviumnetworks.com>
>>>> Reviewed-by: Maxime Coquelin <maxime.coquelin at redhat.com>
>>>> Acked-by: Hemant Agrawal <hemant.agrawal at nxp.com>
>>>> ---
>>>> v6 --> v7:
>>>> - squashed v6 series patch no [01/12] & [05/12]..
>>>>   i.e.. moved RTE_PCI_DRV_IOVA_AS_VA flag into this patch. (Aaron
>>>> comment).
>>>>
>>>>  lib/librte_eal/common/include/rte_pci.h         |  2 +
>>>>  lib/librte_eal/linuxapp/eal/eal_pci.c           | 95
>>>> +++++++++++++++++++++++++
>>>>  lib/librte_eal/linuxapp/eal/eal_vfio.c          | 19 +++++
>>>>  lib/librte_eal/linuxapp/eal/eal_vfio.h          |  4 ++
>>>>  lib/librte_eal/linuxapp/eal/rte_eal_version.map |  1 +
>>>>  5 files changed, 121 insertions(+)
>>>>
>>>> diff --git a/lib/librte_eal/common/include/rte_pci.h
>>>> b/lib/librte_eal/common/include/rte_pci.h
>>>> index 0e36de093..a67d77f22 100644
>>>> --- a/lib/librte_eal/common/include/rte_pci.h
>>>> +++ b/lib/librte_eal/common/include/rte_pci.h
>>>> @@ -202,6 +202,8 @@ struct rte_pci_bus {  #define
>>>> RTE_PCI_DRV_INTR_RMV 0x0010
>>>>  /** Device driver needs to keep mapped resources if unsupported dev
>>>> detected */  #define RTE_PCI_DRV_KEEP_MAPPED_RES 0x0020
>>>> +/** Device driver supports iova as va */ #define
>>>> RTE_PCI_DRV_IOVA_AS_VA
>>>> +0X0040
>>>>
>>>>  /**
>>>>   * A structure describing a PCI mapping.
>>>> diff --git a/lib/librte_eal/linuxapp/eal/eal_pci.c
>>>> b/lib/librte_eal/linuxapp/eal/eal_pci.c
>>>> index 8951ce742..9725fd493 100644
>>>> --- a/lib/librte_eal/linuxapp/eal/eal_pci.c
>>>> +++ b/lib/librte_eal/linuxapp/eal/eal_pci.c
>>>> @@ -45,6 +45,7 @@
>>>>  #include "eal_filesystem.h"
>>>>  #include "eal_private.h"
>>>>  #include "eal_pci_init.h"
>>>> +#include "eal_vfio.h"
>>>>
>>>>  /**
>>>>   * @file
>>>> @@ -487,6 +488,100 @@ rte_pci_scan(void)
>>>>  	return -1;
>>>>  }
>>>>
>>>> +/*
>>>> + * Is pci device bound to any kdrv
>>>> + */
>>>> +static inline int
>>>> +pci_device_is_bound(void)
>>>> +{
>>>> +	struct rte_pci_device *dev = NULL;
>>>> +	int ret = 0;
>>>> +
>>>> +	FOREACH_DEVICE_ON_PCIBUS(dev) {
>>>> +		if (dev->kdrv == RTE_KDRV_UNKNOWN ||
>>>> +		    dev->kdrv == RTE_KDRV_NONE) {
>>>> +			continue;
>>>> +		} else {
>>>> +			ret = 1;
>>>> +			break;
>>>> +		}
>>>> +	}
>>>> +	return ret;
>>>> +}
>>>> +
>>>> +/*
>>>> + * Any one of the device bound to uio  */ static inline int
>>>> +pci_device_bound_uio(void)
>>>> +{
>>>> +	struct rte_pci_device *dev = NULL;
>>>> +
>>>> +	FOREACH_DEVICE_ON_PCIBUS(dev) {
>>>> +		if (dev->kdrv == RTE_KDRV_IGB_UIO ||
>>>> +		   dev->kdrv == RTE_KDRV_UIO_GENERIC) {
>>>> +			return 1;
>>>> +		}
>>>> +	}
>>>> +	return 0;
>>>> +}
>>>> +
>>>> +/*
>>>> + * Any one of the device has iova as va  */ static inline int
>>>> +pci_device_has_iova_va(void)
>>>> +{
>>>> +	struct rte_pci_device *dev = NULL;
>>>> +	struct rte_pci_driver *drv = NULL;
>>>> +
>>>> +	FOREACH_DRIVER_ON_PCIBUS(drv) {
>>>> +		if (drv && drv->drv_flags & RTE_PCI_DRV_IOVA_AS_VA) {
>>>> +			FOREACH_DEVICE_ON_PCIBUS(dev) {
>>>> +				if (dev->kdrv == RTE_KDRV_VFIO &&
>>>> +				    rte_pci_match(drv, dev))
>>>> +					return 1;
>>>> +			}
>>>> +		}
>>>> +	}
>>>> +	return 0;
>>>> +}
>>>> +
>>>> +/*
>>>> + * Get iommu class of PCI devices on the bus.
>>>> + */
>>>> +enum rte_iova_mode
>>>> +rte_pci_get_iommu_class(void)
>>>> +{
>>>> +	bool is_bound;
>>>> +	bool is_vfio_noiommu_enabled = true;
>>>> +	bool has_iova_va;
>>>> +	bool is_bound_uio;
>>>> +
>>>> +	is_bound = pci_device_is_bound();
>>>> +	if (!is_bound)
>>>> +		return RTE_IOVA_DC;
>>>> +
>>>> +	has_iova_va = pci_device_has_iova_va();
>>>> +	is_bound_uio = pci_device_bound_uio(); #ifdef VFIO_PRESENT
>>>> +	is_vfio_noiommu_enabled = vfio_noiommu_is_enabled() == 1 ? 1 :
>>>> 0;
>>> If you specify is_vfio_noiommu_enabled as bool, you should probably treat
>> it as such, and assign true/false.
>>
>> queued for v8.
>>
>>> Other than that, I'm curious why is it always set to "true" by default? If we
>> don't have VFIO compiled, it seems like the error message would always
>> complain about vfio-noiommu mode being enabled, which is confusing.
>>
>> Set to 'true' for case when VFIO_PRESENT unset.. meaning platform doesn't
>> support VFIO (linux versioned < 3.6) i.e.. using UIO - In that case, flag makes
>> sure _pa policy selected.
>>
>> On error message: It won't come in non-vfio case, as 'has_iova_va' will set to
>> 0.
>> Error message will show for those case where few device out of many bind
>> to uio, so message will pop-up and iova policy would be _pa in that case.
>>
>> Thanks.
> Right. My apologies, I misunderstood the meaning of "has_iova_va" flag.

No worry ;). Thanks for review feedback and looking into v7 series.

Can I collect your reviewed-by: for [3/9]?

Thanks. 

> Thanks,
> Anatoly
>
>>> Thanks,
>>> Anatoly



More information about the dev mailing list