[dpdk-dev] [PATCH 0/6] introduce DMA memory mapping for external memory

[Burakov, Anatoly]

On 13-Feb-19 7:24 PM, Shahaf Shuler wrote:
> Wednesday, February 13, 2019 1:43 PM, Alejandro Lucero:
>> Subject: Re: [dpdk-dev] [PATCH 0/6] introduce DMA memory mapping for
>> external memory
>>
>> On Wed, Feb 13, 2019 at 9:11 AM Shahaf Shuler <shahafs at mellanox.com>
>> wrote:
>>
>>> This series is in continue to RFC[1].
>>>
>>> The DPDK APIs expose 3 different modes to work with memory used for
>> DMA:
>>>
>>> 1. Use the DPDK owned memory (backed by the DPDK provided
>> hugepages).
>>> This memory is allocated by the DPDK libraries, included in the DPDK
>>> memory system (memseg lists) and automatically DMA mapped by the
>> DPDK
>>> layers.
>>>
>>> 2. Use memory allocated by the user and register to the DPDK memory
>>> systems. This is also referred as external memory. Upon registration
>>> of the external memory, the DPDK layers will DMA map it to all needed
>>> devices.
>>>
>>> 3. Use memory allocated by the user and not registered to the DPDK
>>> memory system. This is for users who wants to have tight control on
>>> this memory. The user will need to explicitly call DMA map function in
>>> order to register such memory to the different devices.
>>>
>>> The scope of the patch focus on #3 above.
>>>
>>>
>> Why can not we have case 2 covering case 3?
> 
> Because it is not our choice rather the DPDK application.
> We could not allow it, and force the application to register their external memory to the DPDK memory management system. However IMO it will be wrong.
> The use case exists  - some application wants to manage their memory by themselves. w/o the extra overhead of rte_malloc, without creating a special socket to populate the memory and without redundant API calls to rte_extmem_*.
> 
> Simply allocate chunk of memory, DMA map it to device and that’s it.

Just a small note: while this sounds good on paper, i should point out 
that at least *registering* the memory with DPDK is a necessity. You may 
see rte_extmem_* calls as redundant (and i agree, to an extent), but we 
don't advertise our PMD's capabilities in a way that makes it easy to 
determine whether a particular PMD will or will not work without 
registering external memory within DPDK (i.e. does it use 
rte_virt2memseg() internally, for example).

So, extmem register calls are a necessary evil in such case, and IMO 
should be called out as required for such external memory usage scenario.

-- 
Thanks,
Anatoly