Memory management BoF summary (DPDK Summit 2026)

[Mattias Rönnblom]

On 5/18/26 10:35, Morten Brørup wrote:
> There are two ways of allocating/freeing memory objects in DPDK:
> 1.	From the memory heap, i.e. rte_malloc() etc.
> 2.	From a memory pool, i.e. rte_mempool_get() etc.
> 
> The current memory heap in DPDK has two issues:
> -	It is too slow to be used in the fast path.
> -	There may be contention between control threads and
> 	EAL threads for the same spinlocks in the heap.
> 
> The memory pools in the DPDK has multiple issues:
> -	It is designed for fixed size objects,
> 	so individual mempools must be allocated for each type of object.
> 	(In theory, mempools could be allocated per object size,
> 	and shared across modules.
> 	But that would require coordination across modules, including
> 	the total number of objects required by these modules,
> 	the choice of underlying mempool driver,
> 	and the optimal mempool cache size.)
> -	The number of objects in each mempool is fixed,
> 	so applications have to size their mempools to worst case usage.
> -	The memory for the objects in a mempool is pre-allocated.
> In summary, mempools consume significantly more memory than effectively
> being used.
> 
> We discussed the need for a new memory heap system,
> and converged on the following features:
> -	Providing functions to allocate and free memory objects of varying
> 	size, like the rte_malloc library, but usable in the fast path.
> -	Perhaps also a need for bulk alloc/free functions,
> 	like the rte_mempool library.
> -	Building on top of memzones.
> -	No dependency on the current DPDK heap, so it can cleanly replace it.
> 	This is a stretch requirement.
> -	NUMA aware.
> -	Using slabs for various block sizes like in the Linux Kernel,
> 	possibly with object size 2^N.
> -	Using per-lcore caches to reduce contention,
> 	resembling the mempool per-lcore caches.
> 
> Mattias Rönnblom has implemented a prototype with the above properties,
> and this was a good starting point for the discussion.
> 

I'm working to get an OK from the appropriate internal stakeholders, 
then I'll post an RFC.

Thanks a lot for the excellent summary.

> Discussion:
> -	It does not seem to be a requirement to be able to free the memory
> 	used by the heap back to the memzones.
> -	It is possible having header-less objects.
> 	However, if the headers are relatively small compared to the size of
> 	the objects themselves, having a header may offer some benefits.
> -	The mempool library has implementation details optimizing for
> 	spreading usage across memory channels, cache alignment, etc..
> 	Such performance optimization details might need consideration.
> -	There is a lower limit to how small objects will be allocated.
> 	E.g. objects smaller than the size of a pointer is unlikely.
> -	It is acceptable to return an object larger than the size requested,
> 	e.g. returning an object of 16 bytes when 12 bytes were requested.
> -	Preventing false sharing of allocated objects between CPU caches
> 	may not be necessary, but must be considered.
> -	Optimally, the new heap should replace the existing heap.
> 	As seen with the timer wheel library previously proposed, replacing
> 	core libraries in DPDK is difficult, so adding the new heap library
> 	in parallel with the existing heap, i.e. with separate APIs, might be
> 	a path of less resistance.
> 	It will then be an application choice to use this new memory heap.
> 	And long term, it can replace the existing heap, if appropriate.
> 
> Mbuf discussion:
> -	As a stretch goal, mbufs should be dynamically allocated from
> 	the new heap, instead of being pre-allocated in mempools.
> -	This might be achievable either
> 	-	by replacing the mempool library with a wrapper to the heap, or
> 	-	by providing a new mempool driver using the heap, as an
> 		alternative to the existing ring and stack mempool drivers.

Would be very interesting to see a prototype of a mempool driver for a 
new small-object allocator.

> -	In this context, it is important to note that mbuf objects have some
> 	pre-initialized fields when held in their respective mempools.
> 	If allocating an mbuf directly from the heap, these fields must
> 	somehow be initialized.
> 	We did not discuss solutions for this, but noted that using the new
> 	heap for mbufs should be considered in the design choices.
> 
> 
> Venlig hilsen / Kind regards,
> -Morten Brørup