[dpdk-dev] [PATCH] eal: force IOVA mode to physical

[Eric Zhang]

On 09/07/2018 05:26 AM, Burakov, Anatoly wrote:
> On 06-Sep-18 8:34 AM, Jerin Jacob wrote:
>> -----Original Message-----
>>> Date: Tue, 4 Sep 2018 23:40:36 -0400
>>> From: Eric Zhang <eric.zhang at windriver.com>
>>> To: santosh <santosh.shukla at caviumnetworks.com>, 
>>> hemant.agrawal at nxp.com,
>>>   Gaëtan Rivet <gaetan.rivet at 6wind.com>, "Burakov, Anatoly"
>>>   <anatoly.burakov at intel.com>
>>> CC: bruce.richardson at intel.com, dev at dpdk.org, 
>>> Allain.Legacy at windriver.com,
>>>   Matt.Peters at windriver.com
>>> Subject: Re: [dpdk-dev] [PATCH] eal: force IOVA mode to physical
>>> User-Agent: Mozilla/5.0 (X11; Linux x86_64; rv:52.0) Gecko/20100101
>>>   Thunderbird/52.9.1
>>>
>>> On 08/30/2018 08:59 AM, santosh wrote:
>>>> On Thursday 30 August 2018 05:43 PM, Hemant wrote:
>>>>> External Email
>>>>>
>>>>> Hi,
>>>>>
>>>>> On 8/30/2018 3:13 PM, Gaëtan Rivet wrote:
>>>>>> Hi,
>>>>>>
>>>>>> On Thu, Aug 30, 2018 at 10:09:04AM +0100, Burakov, Anatoly wrote:
>>>>>>> On 29-Aug-18 4:58 PM, eric zhang wrote:
>>>>>>>> This patch adds a configuration option to force the IOVA mode to
>>>>>>>> physical address (PA). There exists virtual devices that are not
>>>>>>>> directly attached to the PCI bus, and therefore the auto detection
>>>>>>>> of the IOVA mode based on probing the PCI bus and IOMMU 
>>>>>>>> configuration
>>>>>>>> may not report the required addressing mode. Having the 
>>>>>>>> configuration
>>>>>>>> option permits the mode to be explicitly configured in this 
>>>>>>>> scenario.
>>>>>>>>
>>>>>>>> Signed-off-by: eric zhang <eric.zhang at windriver.com>
>>>>>>>> ---
>>>>>>> Defining this at compile-time seems like an overkill. Wouldn't 
>>>>>>> it be better
>>>>>>> to just add an EAL command-line option to force IOVA mode to a 
>>>>>>> particular
>>>>>>> value?
>>>>> That is a good suggestion.
>>>>>>> -- 
>>>>>>> Thanks,
>>>>>>> Anatoly
>>>>>> What is the bus of these devices and why not implement 
>>>>>> get_iommu_class
>>>>>> in it?
>>>>> There are cases, where you are using dpdk libraries with external
>>>>> libraries and you need to change the default behavior DPDK lib to use
>>>>> physical address instead of virtual address.
>>>>> Providing an option to user will help.
>>>>>
>>>>>
>>>> More appropriate solution could be:
>>>> * Either fix it at bus layer .. i.e.. get_iommu_class()..
>>>> * Or introduce something like [1] --iova-mode=<pa/va> param.
>>>>
>>>> Former is better solution than latter if autodetection is a key 
>>>> criteria.
>>>> Thanks.
>>>>
>>>> [1] http://patchwork.dpdk.org/patch/25192/
>>>>
>>> It's not generic which couldn't be fixed at bus layer.
>>> So what's the preference of EAL option or compile time solution?
>>> Adding --iova-mode as patch [1] will overrivde auto-detection
>>> rte_bus_get_iommu_class()
>>> make it no use; compile time solution will align with upstream and keep
>>> new atuodetection
>>> solution in #ifndef.
>>
>> If it is for vdev devices, why not introduce something like
>> RTE_PCI_DRV_IOVA_AS_VA and let vdev device describe its personality.
>> And based on the devices(flags) on vdev bus, rte_bus_get_iommu_class()
>> of vdev can decide the mode just like PCI bus.
>>
>
> That seems like a better option to me, +1. As far as i know, at the 
> moment if there are no devices attached at all, or if there are only 
> vdev devices attached, DPDK will default to IOVA as PA mode for no 
> good reason; such a change would certainly fix this.
Thanks for the suggestions however our virtual device doesn't run dpdk 
vdev code so we can't use the flag.
Notice that in eal.c there is one workaround that force iova to be PA 
per virtual device is not directly attached to pci. That case is 
checking kni module. Ours is a similar case that virtual device not 
attach pci directly.
So we have to turn to force iova to PA either 1. compilation option 2. 
eal option.  Which one should be the preference by taking into 
consideration that align with upstream?

Thanks

>
>>
>>>
>>> Thanks
>>> Eric
>>>
>>
>
>