[PATCH] config: limit lcore variable maximum size to 4k

[Mattias Rönnblom]

On 2024-11-08 20:53, Morten Brørup wrote:
>> From: Morten Brørup [mailto:mb at smartsharesystems.com]
>> Sent: Friday, 8 November 2024 19.35
>>
>>> From: David Marchand [mailto:david.marchand at redhat.com]
>>> Sent: Friday, 8 November 2024 19.18
>>>
>>> OVS locks all pages to avoid page faults while processing packets.
> 
> It sounds smart, so I just took a look at how it does this. I'm not sure, but it seems like it only locks pages that are actually mapped (current and future).
> 

mlockall(MLOCK_CURRENT) will bring in the whole BSS, it seems. Plus all 
the rest like unused parts of the execution stacks, the data section and 
unused code (text) in the binary and all libraries it has linked to.

It makes a simple (e.g., a unit test) DPDK 24.07 program use ~33x more 
residential memory. After lcore variables, the same MLOCK_CURRENT-ed 
program is ~30% larger than before. So, a relatively modest increase.

The numbers are less drastic, obviously, for many real-world programs, 
which have large packet pools and other memory hogs.

>>> 1M for each lcore translates to allocating 128M with default build
>>> options on x86.
>>> This resulted in OOM while running unit tests in parallel.
> 
> Is the root cause the lcore variables library itself, or the unit test using a lot of memory for testing the lcore variables?
> We don't want to fix the library if the problem is elsewhere.
> 
>>>
>>> At the moment, the more demanding DPDK user of lcore variable is
>>> rte_service, with a 2112 bytes object.
>>>
>>> Limit the lcore variable maximum size to 4k which looks more
>>> reasonable.
>>
>> 4 KB is not future proof.
>>
>> Here's an example where 16 KB is cutting it close:
>> https://inbox.dpdk.org/dev/98CBD80474FA8B44BF855DF32C47DC35E9F7D0@smart
>> server.smartshare.dk/
>>
>> Depends on how we are going to use it. 4 KB suffices if we only want to
>> use it for "small" structures.
>>
>> Would 64 KB work as a compromise?
>