[dpdk-dev] Big spike in DPDK VSZ

[Burakov, Anatoly]

On 30-Jan-20 8:51 AM, David Marchand wrote:
> On Thu, Jan 30, 2020 at 8:48 AM siddarth rai <siddsr at gmail.com> wrote:
>> I have been using DPDK 19.08 and I notice the process VSZ is huge.
>>
>> I tried running the test PMD. It takes 64G VSZ and if I use the
>> '--in-memory' option it takes up to 188G.
>>
>> Is there anyway to disable allocation of such huge VSZ in DPDK ?
> 
> *Disclaimer* I don't know the arcanes of the mem subsystem.
> 
> I suppose this is due to the memory allocator in dpdk that reserves
> unused virtual space (for memory hotplug + multiprocess).

Yes, that's correct. In order to guarantee memory reservation succeeding 
at all times, we need to reserve all possible memory in advance. 
Otherwise we may end up in a situation where primary process has 
allocated a page, but the secondary can't map it because the address 
space is already occupied by something else.

> 
> If this is the case, maybe we could do something to enhance the
> situation for applications that won't care about multiprocess.
> Like inform dpdk that the application won't use multiprocess and skip
> those reservations.

You're welcome to try this, but i assure you, avoiding these 
reservations is a lot of work, because you'd be adding a yet another 
path to an already overly complex allocator :)

> 
> Or another idea would be to limit those reservations to what is passed
> via --socket-limit.
> 
> Anatoly?

I have a patchset in the works that does this and was planning to submit 
it to 19.08, but things got in the way and it's still sitting there 
collecting bit rot. This may be reason enough to resurrect it and finish 
it up :)

> 
> 
> 
> --
> David Marchand
> 


-- 
Thanks,
Anatoly