[PATCH v8 7/9] memarea: support backup memory mechanism

[Mattias Rönnblom]

On 2022-10-11 17:58, Dmitry Kozlyuk wrote:
> 2022-10-11 12:17 (UTC+0000), Chengwen Feng:
>> This patch adds a memarea backup mechanism, where an allocation request
>> which cannot be met by the current memarea is deferred to its backup
>> memarea.
> 
> This is a controversial feature.
> 
> 1. It violates memarea property of freeing all allocated objects
>     at once when the memarea itself is destroyed. Objects allocated
>     in the backup memarea through the destroyed one will remain.
> 
> 2. If there was an API to check that the object belongs to a memarea
>     (the check from rte_memarea_update_refcnt() in this patch),
>     it would be trivial to implement this feature on top of memarea API.
> 
> Nit: "Deferred" is about time -> "forwarded", "delegated", or "handled over".
> 
> A general note about this series.
> IMO, libraries should have limited scope and allow composition
> rather than accumulate features and control their use via options.
> The core idea of memarea is an allocator within a memory region,
> a fast one and with low overhead, usable to free all objects at once.
> 

What's a typical use case for a memory region? In a packet processing 
context.

The ability to instantiate a variable number of heaps/regions seems 
useful, although it's not clear to me if the application should order 
that to happen on a per-lcore basis, on a per-NUMA node basis, a 
per-<some domain object>-basis, or something else entirely.

It seems to me that DPDK is lacking a variable-size memory allocator 
which is efficient and safe to use from lcore threads. My impression is 
that glibc malloc() and rte_malloc() are too slow for the packet 
processing threads, and involves code paths taking locks shared with 
non-EAL threads.

> This is orthogonal to the question from where the memory comes from.
> HEAP and LIBC sources could be built on top of USER source,
> which means that the concept of source is less relevant.
> Backup mechanism could instead be a way to add memory to the area,
> in which case HEAP and LIBC memarea would also be expandable.
> Memarea API could be defined as a structure with callbacks,
> and different types of memarea could be combined,
> for example, interlocked memarea on top of expandable memarea on top of
> memarea with a particular memory management algorithm.
> 
> I'm not saying we should immediately build all this complexity.

The part with implementing runtime polymorphism using a struct with 
function pointers, instead of the enum+switch-based-type-test approach, 
doesn't sound like something that would add complexity. Rather the opposite.

Also, having a clear-cut separation of concern between 
the-thing-that-allocates-and-frees-the-region and the region-internal 
memory manager (what's called an algorithm in this patchset) also seems 
like something that would simplify the code.

> On the contrary, I would merge the basic things first,
> then try to _stack_ new features on top,
> then look if interfaces emerge that can be used for composition.