[dpdk-dev] [PATCH v4 2/4] eal: fix IOVA mode selection as VA for PCI drivers

[Ferruh Yigit]

On 7/22/2019 1:56 PM, David Marchand wrote:
> The incriminated commit broke the use of RTE_PCI_DRV_IOVA_AS_VA which
> was intended to mean "driver only supports VA" but had been understood
> as "driver supports both PA and VA" by most net drivers and used to let
> dpdk processes to run as non root (which do not have access to physical
> addresses on recent kernels).
> 
> The check on physical addresses actually closed the gap for those
> drivers. We don't need to mark them with RTE_PCI_DRV_IOVA_AS_VA and this
> flag can retain its intended meaning.
> Document explicitly its meaning.
> 
> We can check that a driver requirement wrt to IOVA mode is fulfilled
> before trying to probe a device.
> 
> Finally, document the heuristic used to select the IOVA mode and hope
> that we won't break it again.
> 
> Fixes: 703458e19c16 ("bus/pci: consider only usable devices for IOVA mode")
> 
> Signed-off-by: David Marchand <david.marchand at redhat.com>
> Reviewed-by: Jerin Jacob <jerinj at marvell.com>
> Tested-by: Jerin Jacob <jerinj at marvell.com>
> Acked-by: Anatoly Burakov <anatoly.burakov at intel.com>
<...>

> diff --git a/drivers/bus/pci/pci_common.c b/drivers/bus/pci/pci_common.c
> index d2af472..9794552 100644
> --- a/drivers/bus/pci/pci_common.c
> +++ b/drivers/bus/pci/pci_common.c
> @@ -169,8 +169,22 @@ static struct rte_devargs *pci_devargs_lookup(struct rte_pci_device *dev)
>  	 * This needs to be before rte_pci_map_device(), as it enables to use
>  	 * driver flags for adjusting configuration.
>  	 */
> -	if (!already_probed)
> +	if (!already_probed) {
> +		enum rte_iova_mode dev_iova_mode;
> +		enum rte_iova_mode iova_mode;
> +
> +		dev_iova_mode = pci_device_iova_mode(dr, dev);
> +		iova_mode = rte_eal_iova_mode();
> +		if (dev_iova_mode != RTE_IOVA_DC &&
> +		    dev_iova_mode != iova_mode) {
> +			RTE_LOG(ERR, EAL, "  Expecting '%s' IOVA mode but current mode is '%s', not initializing\n",
> +				dev_iova_mode == RTE_IOVA_PA ? "PA" : "VA",
> +				iova_mode == RTE_IOVA_PA ? "PA" : "VA");
> +			return -EINVAL;
> +		}
> +

OvS reported an error while hotplugging a device.

It looks like DPDK application initialized as IOVA=VA, and the new device is bound to 'igb_uio' which forces it to PA, fails on above check.

I would like to get your comment on the issue.

For the OvS mode, hopefully binding the device to 'vfio-pci' can be a solution, but for the cases we don't have that option, can/should we force the DPDK to PA mode after initialization?