[PATCH 0/3] Defer lcore variables allocation

[Mattias Rönnblom]

On 2024-12-05 18:57, David Marchand wrote:
> As I had reported in rc2, the lcore variables allocation have a
> noticeable impact on applications consuming DPDK, even when such
> applications does not use DPDK, or use features associated to
> some lcore variables.
> 
> While the amount has been reduced in a rush before rc2,
> there are still cases when the increased memory footprint is noticed
> like in scaling tests.
> See https://bugs.launchpad.net/ubuntu/+source/dpdk/+bug/2090931
> 

What this bug report fails to mention is that it only affects 
applications using locked memory.

> 
> lcore variable allocations in constructor is a bad idea, as the
> application consuming DPDK has no control over such allocation:
> linking some code does not mean that all of it will be used at runtime.
> 

The core issue isn't really the allocations themselves, or the lack of 
app-level control. Lcore variables are no different from the static 
arrays they are replacing, and existing ones are small enough to be 
pretty harmless.

Such innocent allocations trigger a larger (bulk) allocation, which is 
also harmless, unless you use locked memory.

I was clear on the impact on RSS for locked-memory type apps, but nobody 
pointed this out as an issue worth fixing, so I made no attempt in doing so.

In retrospect, maybe the offset between lcore variable instances could 
have been encoded into the handle, and thus one could use 
different-sized offset for different variables.

> The general question on whether lcore variables in constructor should
> be forbidden, is left to a later discussion.
> 

That discussion could be extended to cover the question if RTE_INIT() 
type constructors should be used at all. Intuitively, it seems better if 
all DPDK initialization, or at least all EAL init, happens at the time 
of rte_eal_init(), in some ordered/organized fashion.

> For now, this series only focus on fixing subsystems using lcore
> variables so that those allocations are deferred either in rte_eal_init()
> or in the path that does require such lcore variables.
> 
>