Need help with reducing VIRT memory

[Bruce Richardson]

On Fri, May 03, 2024 at 04:27:39PM +0100, Bruce Richardson wrote:
> On Fri, May 03, 2024 at 02:48:12PM +0000, Lombardo, Ed wrote:
> > Hi Dmitry,
> > I am not clear on the DPDK memory layout and how to tweak these #define values.  
> > 
> > #define RTE_MAX_MEMSEG_PER_LIST 8192
> > #define RTE_MAX_MEM_MB_PER_LIST 32768
> > #define RTE_MAX_MEMSEG_PER_TYPE 32768
> > #define RTE_MAX_MEM_MB_PER_TYPE 65536
> > 
> > I want to limit how much DPDK grabs for memory, but grabs what it absolutely needs for our application.
> 
> Hi,
> 
> This is what DPDK does. What is being shown in the VIRT figures is the
> address space reservation DPDK has made, but not what memory it actually uses.
> Only sufficient hugepage memory to meet the demands of your app should be
> mapped by DPDK, the rest is unused address space that is not taking up any
> actual memory.
> 
By way of illustration, here is the memory output for a testpmd process on
my system. I got this by running "top -b -p <testpmd-PID>"

    PID USER      PR  NI    VIRT    RES    SHR S  %CPU  %MEM     TIME+ COMMAND
2336969 bruce     20   0  256.2g  26432  19712 S  93.8   0.0   5:28.13 dpdk-testpmd

If we look at the memory relevant columns, indeed VIRT shows a huge value -
256G in my case. However, the actual RAM used by testpmd is given in the
"RES" (resident??) column, showing that testpmd actually is only using
26,432kB of memory in this instance, of which 19,712kB is shared memory
(mostly hugepages). In fact, testpmd actually has even more hugepage memory
than that mapped into it, but they must not be actually in use. [Anatoly,
can you confirm that this would be the case when using vfio-pci i.e. no
physical addresses to query?]

Regards,
/Bruce