[dpdk-dev] [PATCH 00/16] Support externally allocated memory in DPDK

[Burakov, Anatoly]

On 13-Sep-18 8:44 AM, Shahaf Shuler wrote:
> Hi Anatoly,
> 
> First thanks for the patchset, it is a great enhancement.
> 
> See question below.
> 
> Tuesday, September 4, 2018 4:12 PM, Anatoly Burakov:
>> Subject: [dpdk-dev] [PATCH 00/16] Support externally allocated memory in
>> DPDK
>>
>> This is a proposal to enable using externally allocated memory in DPDK.
>>
>> In a nutshell, here is what is being done here:
>>
>> - Index internal malloc heaps by NUMA node index, rather than NUMA
>>    node itself (external heaps will have ID's in order of creation)
>> - Add identifier string to malloc heap, to uniquely identify it
>>    - Each new heap will receive a unique socket ID that will be used by
>>      allocator to decide from which heap (internal or external) to
>>      allocate requested amount of memory
>> - Allow creating named heaps and add/remove memory to/from those
>> heaps
>> - Allocate memseg lists at runtime, to keep track of IOVA addresses
>>    of externally allocated memory
>>    - If IOVA addresses aren't provided, use RTE_BAD_IOVA
>> - Allow malloc and memzones to allocate from external heaps
>> - Allow other data structures to allocate from externall heaps
>>
>> The responsibility to ensure memory is accessible before using it is on the
>> shoulders of the user - there is no checking done with regards to validity of
>> the memory (nor could there be...).
> 
> That makes sense. However who should be in-charge of mapping this memory for dma access?
> The user or internally be the PMD when encounter the first packet or while traversing the existing mempools?
> 
Hi Shahaf,

There are two ways this can be solved. The first way is to perform VFIO 
mapping automatically on adding/attaching memory. The second is to force 
user to do it manually. For now, the latter is chosen because user knows 
best if they intend to do DMA on that memory, but i'm open to suggestions.

There is an issue with some devices and buses (i.e. bus/fslmc) bypassing 
EAL VFIO infrastructure and performing their own VFIO/DMA mapping magic, 
but solving that problem is outside the scope of this patchset. Those 
devices/buses should fix themselves :)

When not using VFIO, it's out of our hands anyway.

-- 
Thanks,
Anatoly