DPDK 22.11.2 requires too much VIRT memory, how to reduce

[Dmitry Kozlyuk]

Hi Ed,

2023-11-10 00:16 (UTC+0000), Lombardo, Ed:
> I finally finished testing all the options and found the VIRT value can be
> reduced from 66 GB to 16 GB with --legacy-mem setting in EAL init argument.

Right.
By default, DPDK can use up to 64 GB of hugepage memory,
so it reserves 64 GB of VIRT (but does not map most of it),
RES should be low until the app actually allocates something.
In legacy mode, DPDK maps all available hugepage memory at startup,
in your case 16 GB, VIRT and RES should be close.

>  So I therefore had to increase the VM memory from 16 GB to 24 GB (instead
> of 80 GB without this setting).

I don't understand why you have to do that.
Possible VIRT is not limited by available RAM.
DPDK should be able to reserve 64 GB of VIRT on a machine with 16 GB of RAM,
it will just be unable to map more than 16 GB (obviously).

> I wonder what do we give up with this setting?
> 
> All the other settings I tried and combinations of these had no impact (socket-limit=2048, single-file-segments, no-shconf, and no-telemetry) on VIRT memory.
> 
> Thanks,
> Ed
> 
> -----Original Message-----
> From: Dmitry Kozlyuk <dmitry.kozliuk at gmail.com> 
> Sent: Wednesday, November 8, 2023 6:06 PM
> To: Lombardo, Ed <Ed.Lombardo at netscout.com>
> Cc: users <users at dpdk.org>
> Subject: Re: DPDK 22.11.2 requires too much VIRT memory, how to reduce
> 
> External Email: This message originated outside of NETSCOUT. Do not click links or open attachments unless you recognize the sender and know the content is safe.
> 
> 2023-11-08 22:26 (UTC+0000), Lombardo, Ed:
> > Hi Dmitry,
> > I went to the dpdk.org and I don’t see DPDK 23.11,
> > 
> > [cid:image001.png at 01DA1268.AA1AC1F0]
> > 
> > Do you have a pre-release version?  
> 
> Sorry for the typo, I meant v22.11.2 that you're testing.
>