[dpdk-dev] Big spike in DPDK VSZ

[siddarth rai]

Hi Anatoly,

I don't need a secondary process.

I tried out Julien's suggestion and set the param 'RTE_MAX_MEM_MB' value to
8192 (the original value was over 500K). This works as a cap.
The virtual size dropped down to less than 8G. So this seems to be working
for me.

I have a few queries/concerns though.
Is it safe to reduce the  RTE_MAX_MEM_MB to such a low value ? Can I reduce
it further ? What will be the impact of doing so ? Will it limit the
maximum size of mbuf pool which I create ?

Regards,
Siddarth

On Tue, Feb 4, 2020 at 3:53 PM Burakov, Anatoly <anatoly.burakov at intel.com>
wrote:

> On 30-Jan-20 8:51 AM, David Marchand wrote:
> > On Thu, Jan 30, 2020 at 8:48 AM siddarth rai <siddsr at gmail.com> wrote:
> >> I have been using DPDK 19.08 and I notice the process VSZ is huge.
> >>
> >> I tried running the test PMD. It takes 64G VSZ and if I use the
> >> '--in-memory' option it takes up to 188G.
> >>
> >> Is there anyway to disable allocation of such huge VSZ in DPDK ?
> >
> > *Disclaimer* I don't know the arcanes of the mem subsystem.
> >
> > I suppose this is due to the memory allocator in dpdk that reserves
> > unused virtual space (for memory hotplug + multiprocess).
>
> Yes, that's correct. In order to guarantee memory reservation succeeding
> at all times, we need to reserve all possible memory in advance.
> Otherwise we may end up in a situation where primary process has
> allocated a page, but the secondary can't map it because the address
> space is already occupied by something else.
>
> >
> > If this is the case, maybe we could do something to enhance the
> > situation for applications that won't care about multiprocess.
> > Like inform dpdk that the application won't use multiprocess and skip
> > those reservations.
>
> You're welcome to try this, but i assure you, avoiding these
> reservations is a lot of work, because you'd be adding a yet another
> path to an already overly complex allocator :)
>
> >
> > Or another idea would be to limit those reservations to what is passed
> > via --socket-limit.
> >
> > Anatoly?
>
> I have a patchset in the works that does this and was planning to submit
> it to 19.08, but things got in the way and it's still sitting there
> collecting bit rot. This may be reason enough to resurrect it and finish
> it up :)
>
> >
> >
> >
> > --
> > David Marchand
> >
>
>
> --
> Thanks,
> Anatoly
>