[dpdk-dev] [RFC v2 00/23] Dynamic memory allocation for DPDK

Burakov, Anatoly anatoly.burakov at intel.com
Tue Dec 19 17:09:41 CET 2017


On 19-Dec-17 4:06 PM, Stephen Hemminger wrote:
> On Tue, 19 Dec 2017 16:02:51 +0000
> "Burakov, Anatoly" <anatoly.burakov at intel.com> wrote:
> 
>> On 19-Dec-17 3:46 PM, Stephen Hemminger wrote:
>>> On Tue, 19 Dec 2017 11:14:27 +0000
>>> Anatoly Burakov <anatoly.burakov at intel.com> wrote:
>>>    
>>>> This patchset introduces a prototype implementation of dynamic memory allocation
>>>> for DPDK. It is intended to start a conversation and build consensus on the best
>>>> way to implement this functionality. The patchset works well enough to pass all
>>>> unit tests, and to work with traffic forwarding, provided the device drivers are
>>>> adjusted to ensure contiguous memory allocation where it matters.
>>>
>>>
>>> What exact functionality is this patchset trying to enable.
>>> It isn't clear what is broken now. Is it a cleanup or something that
>>> is being motivated by memory layout issues?
>>>    
>>
>> Hi Stephen,
>>
>> Apologies for not making that clear enough in the cover letter.
>>
>> The big issue this patchset is trying to solve is the static-ness of
>> DPDK's memory allocation. I.e. you reserve memory on startup, and that's
>> it - you can't allocate any more memory from the system, and you can't
>> free it back without stopping the application.
>>
>> With this patchset, you can do exactly that. You can basically start
>> with zero memory preallocated, and allocate (and free) as you go. For
>> example, if you apply this patchset and run malloc autotest, after
>> startup you will have used perhaps a single 2MB page. While the test is
>> running, you are going to allocate something to the tune of 14MB per
>> socket, and at the end you're back at eating 2MB of hugepage memory,
>> while all of the memory you used for autotest will be freed back to the
>> system. That's the main use case this patchset is trying to address.
>>
>> Down the line, there are other issues to be solved, which are outlined
>> in the cover letter (the aforementioned "discussion points"), but for
>> this iteration, dynamic allocation/free of DPDK memory is the one issue
>> that is being addressed.
>>
> 
> Ok, maybe name it "memory hot add/remove" since dynamic memory allocation
> to me implies redoing malloc.
> 

Well, it _kind of_ redoes malloc in the process as we need to handle 
holes in the address space, but sure, something like "memory hotplug" 
would've perhaps been a more suitable name. Thanks for your feedback, it 
will certainly be taken into account for when an inevitable v1 comes :)

-- 
Thanks,
Anatoly


More information about the dev mailing list