[dpdk-dev] [PATCH] eal: force IOVA mode to physical

[Eric Zhang]

On 09/07/2018 04:13 PM, Eric Zhang wrote:
>
>
> On 09/07/2018 05:26 AM, Burakov, Anatoly wrote:
>> On 06-Sep-18 8:34 AM, Jerin Jacob wrote:
>>> -----Original Message-----
>>>> Date: Tue, 4 Sep 2018 23:40:36 -0400
>>>> From: Eric Zhang <eric.zhang at windriver.com>
>>>> To: santosh <santosh.shukla at caviumnetworks.com>, 
>>>> hemant.agrawal at nxp.com,
>>>>   Gaëtan Rivet <gaetan.rivet at 6wind.com>, "Burakov, Anatoly"
>>>>   <anatoly.burakov at intel.com>
>>>> CC: bruce.richardson at intel.com, dev at dpdk.org, 
>>>> Allain.Legacy at windriver.com,
>>>>   Matt.Peters at windriver.com
>>>> Subject: Re: [dpdk-dev] [PATCH] eal: force IOVA mode to physical
>>>> User-Agent: Mozilla/5.0 (X11; Linux x86_64; rv:52.0) Gecko/20100101
>>>>   Thunderbird/52.9.1
>>>>
>>>> On 08/30/2018 08:59 AM, santosh wrote:
>>>>> On Thursday 30 August 2018 05:43 PM, Hemant wrote:
>>>>>> External Email
>>>>>>
>>>>>> Hi,
>>>>>>
>>>>>> On 8/30/2018 3:13 PM, Gaëtan Rivet wrote:
>>>>>>> Hi,
>>>>>>>
>>>>>>> On Thu, Aug 30, 2018 at 10:09:04AM +0100, Burakov, Anatoly wrote:
>>>>>>>> On 29-Aug-18 4:58 PM, eric zhang wrote:
>>>>>>>>> This patch adds a configuration option to force the IOVA mode to
>>>>>>>>> physical address (PA). There exists virtual devices that are not
>>>>>>>>> directly attached to the PCI bus, and therefore the auto 
>>>>>>>>> detection
>>>>>>>>> of the IOVA mode based on probing the PCI bus and IOMMU 
>>>>>>>>> configuration
>>>>>>>>> may not report the required addressing mode. Having the 
>>>>>>>>> configuration
>>>>>>>>> option permits the mode to be explicitly configured in this 
>>>>>>>>> scenario.
>>>>>>>>>
>>>>>>>>> Signed-off-by: eric zhang <eric.zhang at windriver.com>
>>>>>>>>> ---
>>>>>>>> Defining this at compile-time seems like an overkill. Wouldn't 
>>>>>>>> it be better
>>>>>>>> to just add an EAL command-line option to force IOVA mode to a 
>>>>>>>> particular
>>>>>>>> value?
>>>>>> That is a good suggestion.
>>>>>>>> -- 
>>>>>>>> Thanks,
>>>>>>>> Anatoly
>>>>>>> What is the bus of these devices and why not implement 
>>>>>>> get_iommu_class
>>>>>>> in it?
>>>>>> There are cases, where you are using dpdk libraries with external
>>>>>> libraries and you need to change the default behavior DPDK lib to 
>>>>>> use
>>>>>> physical address instead of virtual address.
>>>>>> Providing an option to user will help.
>>>>>>
>>>>>>
>>>>> More appropriate solution could be:
>>>>> * Either fix it at bus layer .. i.e.. get_iommu_class()..
>>>>> * Or introduce something like [1] --iova-mode=<pa/va> param.
>>>>>
>>>>> Former is better solution than latter if autodetection is a key 
>>>>> criteria.
>>>>> Thanks.
>>>>>
>>>>> [1] http://patchwork.dpdk.org/patch/25192/
>>>>>
>>>> It's not generic which couldn't be fixed at bus layer.
>>>> So what's the preference of EAL option or compile time solution?
>>>> Adding --iova-mode as patch [1] will overrivde auto-detection
>>>> rte_bus_get_iommu_class()
>>>> make it no use; compile time solution will align with upstream and 
>>>> keep
>>>> new atuodetection
>>>> solution in #ifndef.
>>>
>>> If it is for vdev devices, why not introduce something like
>>> RTE_PCI_DRV_IOVA_AS_VA and let vdev device describe its personality.
>>> And based on the devices(flags) on vdev bus, rte_bus_get_iommu_class()
>>> of vdev can decide the mode just like PCI bus.
>>>
>>
>> That seems like a better option to me, +1. As far as i know, at the 
>> moment if there are no devices attached at all, or if there are only 
>> vdev devices attached, DPDK will default to IOVA as PA mode for no 
>> good reason; such a change would certainly fix this.
> Thanks for the suggestions however our virtual device doesn't run dpdk 
> vdev code so we can't use the flag.
> Notice that in eal.c there is one workaround that force iova to be PA 
> per virtual device is not directly attached to pci. That case is 
> checking kni module. Ours is a similar case that virtual device not 
> attach pci directly.
> So we have to turn to force iova to PA either 1. compilation option 2. 
> eal option.  Which one should be the preference by taking into 
> consideration that align with upstream?
>
> Thanks
>
Any comments?
>>
>>>
>>>>
>>>> Thanks
>>>> Eric
>>>>
>>>
>>
>>
>