[dpdk-dev] [PATCH v4] eal: pick IOVA as PA if IOMMU is not available

[Burakov, Anatoly]

On 29-Jul-19 10:31 AM, David Marchand wrote:
> On Fri, Jul 26, 2019 at 5:37 PM Anatoly Burakov
> <anatoly.burakov at intel.com> wrote:
>>
>> When IOMMU is not available, /sys/kernel/iommu_groups will not be
>> populated. This is happening since at least 3.6 when VFIO support
>> was added. If the directory is empty, EAL should not pick IOVA as
>> VA as the default IOVA mode.
>>
>> Signed-off-by: Anatoly Burakov <anatoly.burakov at intel.com>
>> Tested-by: Darek Stojaczyk <dariusz.stojaczyk at intel.com>
>> Tested-by: Jerin Jacob <jerinj at marvell.com>
>> Reviewed-by: Jerin Jacob <jerinj at marvell.com>
>> ---
>>
>> Notes:
>>      v4:
>>      - Fix indentation in release notes' known issues
>>
>>      v3:
>>      - Add documentation changes
>>      - Fix a typo pointed out by checkpatch
>>
>>      v2:
>>      - Decouple IOMMU from VFIO
>>      - Add a check for physical addresses availability
>>
>>   .../prog_guide/env_abstraction_layer.rst      | 27 +++++++++++------
>>   doc/guides/rel_notes/known_issues.rst         | 26 ++++++++++++++++
>>   doc/guides/rel_notes/release_19_08.rst        | 16 ++++++++++
>>   lib/librte_eal/linux/eal/eal.c                | 21 +++++++++++--
>>   lib/librte_eal/linux/eal/eal_vfio.c           | 30 +++++++++++++++++++
>>   lib/librte_eal/linux/eal/eal_vfio.h           |  2 ++
>>   6 files changed, 111 insertions(+), 11 deletions(-)
>>
>> diff --git a/doc/guides/prog_guide/env_abstraction_layer.rst b/doc/guides/prog_guide/env_abstraction_layer.rst
>> index 1487ea550..e6e70e5a8 100644
>> --- a/doc/guides/prog_guide/env_abstraction_layer.rst
>> +++ b/doc/guides/prog_guide/env_abstraction_layer.rst
>> @@ -425,6 +425,9 @@ IOVA Mode Detection
>>   IOVA Mode is selected by considering what the current usable Devices on the
>>   system require and/or support.
>>
>> +On FreeBSD, RTE_IOVA_VA mode is not supported, so RTE_IOVA_PA is always used.
> 
> We still allow setting it via --iova-mode=
> Is it really unsupported ? vdev like rings could work.

Oh, right, we don't *really* support IOVA as VA mode, but you can still 
run in --no-huge mode (or using only virtual devices) and pretend like 
it works ;)

Still, i think FreeBSD should default to PA unless it's not running as root.

> 
> 
>> +On Linux, the IOVA mode is detected based on a heuristic.
>> +
>>   Below is the 2-step heuristic for this choice.
> 
> We can combine those two sentences as a single one.

Sure.

> 
> 
>>
>>   For the first step, EAL asks each bus its requirement in terms of IOVA mode
>> @@ -438,20 +441,26 @@ and decides on a preferred IOVA mode.
>>     RTE_IOVA_VA), then the preferred IOVA mode is RTE_IOVA_DC (see below with the
>>     check on Physical Addresses availability),
>>
>> +If the buses have expressed no preference on which IOVA mode to pick, then a
>> +default is selected using the following logic:
>> +

<snip>

>> @@ -1061,8 +1061,25 @@ rte_eal_init(int argc, char **argv)
>>                  enum rte_iova_mode iova_mode = rte_bus_get_iommu_class();
>>
>>                  if (iova_mode == RTE_IOVA_DC) {
>> -                       iova_mode = RTE_IOVA_VA;
>> -                       RTE_LOG(DEBUG, EAL, "Buses did not request a specific IOVA mode, select IOVA as VA mode.\n");
>> +                       RTE_LOG(DEBUG, EAL, "Buses did not request a specific IOVA mode.\n");
>> +
>> +                       if (!phys_addrs) {
>> +                               /* if we have no access to physical addresses,
>> +                                * pick IOVA as VA mode.
>> +                                */
>> +                               iova_mode = RTE_IOVA_VA;
>> +                               RTE_LOG(DEBUG, EAL, "Physical addresses are unavailable, selecting IOVA as VA mode.\n");
>> +                       } else if (vfio_iommu_enabled()) {
> 
> How about:
> s/vfio_iommu_enabled/is_iommu_available/
> 
> And the code would move from vfio specific files to eal.c.

Can do.

-- 
Thanks,
Anatoly