[RFC v2 2/3] lib: add fastmem library

[Mattias Rönnblom]

On 5/27/26 13:17, Morten Brørup wrote:
>> From: Bruce Richardson [mailto:bruce.richardson at intel.com]
>> Sent: Wednesday, 27 May 2026 12.18
>>
>> On Wed, May 27, 2026 at 12:12:19PM +0200, Mattias Rönnblom wrote:
>>> On 5/26/26 15:23, Stephen Hemminger wrote:
>>>> On Tue, 26 May 2026 10:57:42 +0200
>>>> Mattias Rönnblom <hofors at lysator.liu.se> wrote:
>>>>
>>>>> +__rte_experimental
>>>>> +void *
>>>>> +rte_fastmem_alloc(size_t size, size_t align, unsigned int flags)
>>>>> +	__rte_alloc_size(1) __rte_alloc_align(2);
>>>>
>>>> Should also add attribute __rte_malloc which tells compiler
>>>> that pointer returned cannot alias other memory
>>>>
>>>> And add __rte_dealloc(rte_fastmem_free, 1)
>>>> which tells compiler that the returned pointer should only go
>>>> back to fastmem (not free, rte_free, etc).
>>>
>>> Done. Only works for the single-object ops (not bulk) though.
>>>
>>> I've had a look at how to extend fastmem to support larger
>> allocations
>>> (without suggesting this is the way to go).
>>>
>>> Seems to me that implementation should be something like
>>> a) slab allocator for small objects.
>>> b) a cache-less per-socket page run allocator for mid-sized objects.
>>> c) per-object memzones for large objects.
> 
> All allocations through the library should somehow build on memzones.
> If an application wants "normal" memory, it can use libc free()/malloc().
> 

All allocations come out of memzones. In the c) case, each object is a 
memzone. There is a very limited number of memzones, so they can't be many.

>>>
>>> If one would implement that, you would essentially have a plug-in
>>> replacement for rte_malloc.h (maybe minus some debug and some more
>> esoteric
>>> DPDK heap features).
>>>
>>> Should fastmem be an outright replacement, or something that at least
>>> initially lives alongside the regular heap, maybe with a run- or
>>> compile-time option to make rte_malloc.h functions delegate to
>> fastmem? This
>>> is unclear to me at this point. I fear the more ambitious, cleaner
>> and more
>>> risky DPDK heap replacement path will go they way my attempts to
>> replace
>>> rte_memcpy or rte_timer went.
>>>
>>> I would agree with anyone saying that we should have only one heap-
>> like API
>>> for memory allocations. rte_malloc.h obviously needs to stay, for
>> backward
>>> compatibility reason, if nothing else. I would like to add bulk
>> alloc/free,
>>> and allow for smaller alignments than 64, since slabs can do that
>>> efficiently (DPDK heap per-object header is 128 bytes!). One could
>> either go
>>> about that by extending rte_malloc.h or deprecating that API and
>> starting
>>> anew. In the latter case, one could do many more minor tweaks, like
>> removing
>>> the type pointers (only a nuance), remove the validate function,
>> change and
>>> extend the stats interface, etc.
>>>
>> +1 for replacing rte_malloc. For a replacement, I'd tend towards aiming
>> for
>> compatibilty over trying to fix too many little things at once. While
>> changing a couple of things is ok, I'd rather not force applications to
>> make too many updates to their code when moving from one DPDK version
>> to
>> another.
>>
>> /Bruce
> 
> +1 for taking the path that begins with starting anew.
> The new heap library can be designed specifically for the fast path, based on years of DPDK experience with what the fast path really needs.
> It will allow much broader experimentation along the way.
> We can easily get rid of all slow path legacy stuff in rte_malloc, like the "type" parameter.
> 
> I think we all agree that replacing rte_malloc should be the end game.
> But IMO, it very important that the properties of rte_malloc do not impose any limits on the new heap library.
> 
> Someday, when the new library has sufficiently matured, we can discuss how it can replace the rte_malloc library.
> Maybe some parts of rte_malloc are not replaceable, and need to be deprecated and removed.
> Maybe we all have switched to using the new heap, and nobody is using rte_malloc anymore, so it can simply be removed. ;-)
> 

It would be useful to have a run- or compile-time switch to use fastmem 
instead of the regular DPDK heap. In such a case, having it as an 
external library will complicate things during initialization, if you 
want all allocations to end up in fastmem.

Unfortunately, if a rte_malloc replacement, I think it belongs in the 
EAL, just like today's heap. That doesn't mean it have to start in the EAL.

> Suggestion regarding naming:
> The prefix could be "rte_mem_" instead of "rte_fastmem_" - it is shorter, and most libraries are "fast".
> And then it could live in /lib/memory instead of /lib/fastmem.
> 

mem is very generic, but point taken. Name should change, unclear to 
what. Depends on the role it will serve.