[dpdk-dev] Question about enable CBDMA with ioat PMD ?
Burakov, Anatoly
anatoly.burakov at intel.com
Wed Apr 14 17:33:10 CEST 2021
On 14-Apr-21 2:45 PM, Liang Ma wrote:
> On Wed, Apr 14, 2021 at 02:21:53PM +0100, Bruce Richardson wrote:
>> On Wed, Apr 14, 2021 at 01:52:14PM +0100, Liang Ma wrote:
>>> On Wed, Apr 14, 2021 at 11:48:03AM +0100, Bruce Richardson wrote:
>>>> On Wed, Apr 14, 2021 at 11:37:48AM +0100, Liang Ma wrote:
>>>>> On Tue, Apr 13, 2021 at 07:24:41PM +0100, Bruce Richardson wrote:
>>>>>> On Tue, Apr 13, 2021 at 06:36:38PM +0100, Liang Ma wrote:
>>>>>>> Hi Bruce,
>>>>>>> I look into current IOAT PMD. I found that looks only support the
>>>>>>> latest DSA device which is part of SPR(Please correct me if I'm wrong).
>>>>>>> However, there still is lots existing CPU that support previous CBDMA
>>>>>>> Intel Quick-DATA technology. SPDK enable those device with uio.
>>>>>>> Any hint to enable those CBDMA device with IOAT PMD ?
>>>>>>>
>>>>>>
>>>>>> Hi Liang,
>>>>>>
>>>>>> These devices are still supported by the driver. In the latest releases the
>>>>>> new device support was added but no older device support was removed.
>>>>>> Please let us know if you do hit any problems with running it on the CBDMA
>>>>>> HW devices, because it should all still be working as before.
>>>>>>
>>>>>> Regards,
>>>>>> /Bruce
>>>>> Hi Bruce,
>>>>> I hit a problem with CBDMA enabling.
>>>>>
>>>>> 1. I follow the DPDK IOAT driver Doc, I bind 0000.04.0 device with
>>>>> dpdk-devbind.py
>>>>>
>>>>> 2. I run dpdk-test without any parameter then dpdk-test complain with
>>>>>
>>>>> EAL: VFIO support initialized
>>>>> EAL: 0000:00:04.0 VFIO group is not viable! Not all devices in IOMMU
>>>>> group bound to VFIO or unbound
>>>>> Requested device 0000:00:04.0 cannot be used
>>>>>
>>>>> 3. kernel boot parameter include the "intel_iommu=on"
>>>>>
>>>>> If I still need some extra iommu settings for CBDMA device ?
>>>>>
>>>> It's an IOMMU limitation, not just for CBDMA but for other devices like
>>>> NICs too - you need to ensure all subfunctions on a device are all bound to
>>>> the same IOMMU group. So for the CBDMA instance, to use one channel with
>>>> vfio, the other channels on the device also need to be bound to vfio, or
>>>> else not bound to any driver. If bound to the kernel driver, there will be
>>>> IOMMU conflicts as we can't split the channels between kernel and userspace
>>>> address spaces.
>>>>
>>>> /Bruce
>>> I bind whole device from 00:04.0 to 00:04.7. ioat PMD complain
>>> with "ioat_rawdev_create(): ioat_rawdev_create: Channel appears
>>> locked". Any hint?
>>> Regards
>>> Liang
>>
>> That should be a warning only, I think and the device should still be
>> available. Is that not the case, or is there a later error?
>>
>> /Bruce
> Good news, that's a warning only message, I checked the code.
> after probe the device, rawdev_autotest all passed !
> Many thanks for your help !
> I strongly recommend to add the IOMMU hint of CBDMA(bind whole device)
> into the Doc. I think DSA device don't need to do that right ?
> Best Regards
> Liang
>
This is documented in our VFIO docs [1].
[1]
http://doc.dpdk.org/guides/linux_gsg/linux_drivers.html#binding-and-unbinding-network-ports-to-from-the-kernel-modules
(see Warning section)
--
Thanks,
Anatoly
More information about the dev
mailing list