<html xmlns:v="urn:schemas-microsoft-com:vml" xmlns:o="urn:schemas-microsoft-com:office:office" xmlns:w="urn:schemas-microsoft-com:office:word" xmlns:m="http://schemas.microsoft.com/office/2004/12/omml" xmlns="http://www.w3.org/TR/REC-html40"> <head> <meta http-equiv="Content-Type" content="text/html; charset=utf-8"> <meta name="Generator" content="Microsoft Word 15 (filtered medium)"> <style></style> </head> <body lang="EN-US" link="blue" vlink="purple" style="word-wrap:break-word"> <div class="WordSection1"> Hi Demitry,<o:p></o:p> Thank you for your reply. <o:p></o:p> <o:p> </o:p> Tried many things with no success.<o:p></o:p> I have tried 1G hugepage size x 2 pages and the VIRT value went up from 66.0 G to 128G.<o:p></o:p> I used DPDK 17.11 with 1024 Hugepages of size 2MB and did not see VIRT more than 5.5G.<o:p></o:p> <o:p> </o:p> With 1G hugepage size:<o:p></o:p> [root@vstreamOR9-160 ~]# ./dpdk-hugepages.py --show<o:p></o:p> Node Pages Size Total<o:p></o:p> 0 2 1Gb 2Gb<o:p></o:p> <o:p> </o:p> Hugepages mounted on /dev/hugepages /mnt/huge<o:p></o:p> <o:p> </o:p> You can see the 16 G and 32 G mappings with 1G hugepage size. Any additional mmap we do in our application fails for “OUT OF MEMORY (12)”<o:p></o:p> pmap -d <appPID><o:p></o:p> 0000000100035000 4 rw-s- 0000000000000000 000:00019 fbarray_memseg-1048576k-0-0<o:p></o:p> 0000000140000000 1048576 rw-s- 0000000000000000 000:00027 rtemap_0<o:p></o:p> 0000000180000000 1048576 rw-s- 0000000000000000 000:00027 rtemap_1<o:p></o:p> 00000001c0000000 31457280 ----- 0000000000000000 000:00000 [ anon ]<o:p></o:p> 0000000940000000 4 rw-s- 0000000000000000 000:00019 fbarray_memseg-1048576k-0-1<o:p></o:p> 0000000980000000 33554432 ----- 0000000000000000 000:00000 [ anon ]<o:p></o:p> 0000001180000000 388 rw-s- 0000000000000000 000:00019 fbarray_memseg-2048k-0-0<o:p></o:p> 0000001180200000 16777216 ----- 0000000000000000 000:00000 [ anon ]<o:p></o:p> 0000001580200000 388 rw-s- 0000000000000000 000:00019 fbarray_memseg-2048k-0-1<o:p></o:p> 0000001580400000 16777216 ----- 0000000000000000 000:00000 [ anon ]<o:p></o:p> 0000001980400000 388 rw-s- 0000000000000000 000:00019 fbarray_memseg-2048k-0-2<o:p></o:p> 0000001980600000 16777216 ----- 0000000000000000 000:00000 [ anon ]<o:p></o:p> 0000001d80600000 388 rw-s- 0000000000000000 000:00019 fbarray_memseg-2048k-0-3<o:p></o:p> 0000001d80800000 16777216 ----- 0000000000000000 000:00000 [ anon ]<o:p></o:p> <o:p> </o:p> I tried dpdk_testpmd (# ./dpdk-testpmd -l 0-3 -n 1 -- -i --portmask=0x01) on my VM and the VIRT memory jumps to 66 GB with 2MB hugepage size and 1024 huge pages.<o:p></o:p> <o:p> </o:p> I also tried adding –single-file-segments in the EAL init argument.<o:p></o:p> <o:p> </o:p> When you say VIRT memory is not important then what would you suggest I focus on to determine the resource hog that DPDK 22.11 injects in our application?<o:p></o:p> Is there too many mmaps done by DPDK in 22.11?<o:p></o:p> <o:p> </o:p> Thanks,<o:p></o:p> Ed<o:p></o:p> <o:p> </o:p> <div style="border:none;border-top:solid #E1E1E1 1.0pt;padding:3.0pt 0in 0in 0in"> From: Dmitry Kozlyuk <dmitry.kozliuk@gmail.com> Sent: Sunday, November 5, 2023 1:23 PM To: Lombardo, Ed <Ed.Lombardo@netscout.com> Cc: users <users@dpdk.org> Subject: Re: DPDK 22.11.2 requires too much VIRT memory, how to reduce<o:p></o:p> </div> <o:p> </o:p> <div> <table class="MsoNormalTable" border="0" cellspacing="0" cellpadding="0" width="100%" style="width:100.0%;background:#006068"> <tbody> <tr> <td style="padding:3.75pt 3.75pt 3.75pt 3.75pt"> External Email: This message originated outside of NETSCOUT. Do not click links or open attachments unless you recognize the sender and know the content is safe.<o:p></o:p> </td> </tr> </tbody> </table> </div> <div> <div> Hi Ed, <o:p></o:p> </div> <div> <o:p> </o:p> </div> <div> DPDK indeed reserves a large amount of virtual address space at startup. This does not prevent DPDK from running on low-memory systems, since this reservation does not consume physical memory. So the large figure in VIRT is not an issue. Hugepages are mapped on demand. Each mapped hugepage consunes a file descriptor. If you use 2MB hugepages, the number of open files can exhaust the limit easily. If you don't want to increase it, please see: <a href="https://urldefense.com/v3/__https:/doc.dpdk.org/guides/prog_guide/env_abstraction_layer.html*segment-file-descriptors__;Iw!!Nzg7nt7_!C6krMpuvUwO2CGGJ-X-hjU7gIsnPFVpoBW1CyXcNJrZAWfOP_oODYN5YQ1v0y7yy2Ac2ej2uicsBFMNaZYgBTmXOiOM$">https://doc.dpdk.org/guides/prog_guide/env_abstraction_layer.html#segment-file-descriptors</a><o:p></o:p> </div> </div> </div> </body> </html>