[dpdk-dev] mmap fails with more than 40000 hugepages

De Lara Guarch, Pablo pablo.de.lara.guarch at intel.com
Fri Feb 6 11:26:47 CET 2015



> -----Original Message-----
> From: dev [mailto:dev-bounces at dpdk.org] On Behalf Of Linhaifeng
> Sent: Friday, February 06, 2015 2:04 AM
> To: Damjan Marion (damarion); dev at dpdk.org
> Subject: Re: [dpdk-dev] mmap fails with more than 40000 hugepages
> 
> 
> 
> On 2015/2/5 20:00, Damjan Marion (damarion) wrote:
> > Hi,
> >
> > I have system with 2 NUMA nodes and 256G RAM total. I noticed that DPDK
> crashes in rte_eal_init()
> > when number of available hugepages is around 40000 or above.
> > Everything works fine with lower values (i.e. 30000).
> >
> > I also tried with allocating 40000 on node0 and 0 on node1, same crash
> happens.
> >
> >
> > Any idea what might be causing this?
> >
> > Thanks,
> >
> > Damjan
> >
> 
> cat /proc/sys/vm/max_map_count

I just checked on my board, and having 40k hugepages, you need that value to be over the double of that (a value of 81k should work).

Let us know if that fixed the problem!

Pablo
> 
> 
> >
> > $ cat /sys/devices/system/node/node[01]/hugepages/hugepages-
> 2048kB/nr_hugepages
> > 20000
> > 20000
> >
> > $ grep -i huge /proc/meminfo
> > AnonHugePages:    706560 kB
> > HugePages_Total:   40000
> > HugePages_Free:    40000
> > HugePages_Rsvd:        0
> > HugePages_Surp:        0
> > Hugepagesize:       2048 kB
> >
> >
> > $ sudo ~/src/dpdk/x86_64-native-linuxapp-gcc/app/testpmd -l 5-7 -n 3 --
> socket-mem 512,512
> > EAL: Detected lcore 0 as core 0 on socket 0
> > EAL: Detected lcore 1 as core 1 on socket 0
> > EAL: Detected lcore 2 as core 2 on socket 0
> > EAL: Detected lcore 3 as core 3 on socket 0
> > EAL: Detected lcore 4 as core 4 on socket 0
> > EAL: Detected lcore 5 as core 5 on socket 0
> > EAL: Detected lcore 6 as core 6 on socket 0
> > EAL: Detected lcore 7 as core 7 on socket 0
> > EAL: Detected lcore 8 as core 0 on socket 1
> > EAL: Detected lcore 9 as core 1 on socket 1
> > EAL: Detected lcore 10 as core 2 on socket 1
> > EAL: Detected lcore 11 as core 3 on socket 1
> > EAL: Detected lcore 12 as core 4 on socket 1
> > EAL: Detected lcore 13 as core 5 on socket 1
> > EAL: Detected lcore 14 as core 6 on socket 1
> > EAL: Detected lcore 15 as core 7 on socket 1
> > EAL: Detected lcore 16 as core 0 on socket 0
> > EAL: Detected lcore 17 as core 1 on socket 0
> > EAL: Detected lcore 18 as core 2 on socket 0
> > EAL: Detected lcore 19 as core 3 on socket 0
> > EAL: Detected lcore 20 as core 4 on socket 0
> > EAL: Detected lcore 21 as core 5 on socket 0
> > EAL: Detected lcore 22 as core 6 on socket 0
> > EAL: Detected lcore 23 as core 7 on socket 0
> > EAL: Detected lcore 24 as core 0 on socket 1
> > EAL: Detected lcore 25 as core 1 on socket 1
> > EAL: Detected lcore 26 as core 2 on socket 1
> > EAL: Detected lcore 27 as core 3 on socket 1
> > EAL: Detected lcore 28 as core 4 on socket 1
> > EAL: Detected lcore 29 as core 5 on socket 1
> > EAL: Detected lcore 30 as core 6 on socket 1
> > EAL: Detected lcore 31 as core 7 on socket 1
> > EAL: Support maximum 128 logical core(s) by configuration.
> > EAL: Detected 32 lcore(s)
> > EAL: VFIO modules not all loaded, skip VFIO support...
> > EAL: Setting up memory...
> > EAL: Ask a virtual area of 0x800000 bytes
> > EAL: Virtual area found at 0x7fae2a200000 (size = 0x800000)
> > EAL: Ask a virtual area of 0x7600000 bytes
> > EAL: Virtual area found at 0x7fae22a00000 (size = 0x7600000)
> > EAL: Ask a virtual area of 0x1400000 bytes
> > EAL: Virtual area found at 0x7fae21400000 (size = 0x1400000)
> > EAL: Ask a virtual area of 0x200000 bytes
> > EAL: Virtual area found at 0x7fae21000000 (size = 0x200000)
> > EAL: Ask a virtual area of 0x400000 bytes
> > EAL: Virtual area found at 0x7fae20a00000 (size = 0x400000)
> > EAL: Ask a virtual area of 0x200000 bytes
> > EAL: Virtual area found at 0x7fae20600000 (size = 0x200000)
> > EAL: Ask a virtual area of 0x200000 bytes
> > EAL: Virtual area found at 0x7fae20200000 (size = 0x200000)
> > EAL: Ask a virtual area of 0x200000 bytes
> > EAL: Virtual area found at 0x7fae1fe00000 (size = 0x200000)
> > EAL: Ask a virtual area of 0x5800000 bytes
> > EAL: Virtual area found at 0x7fae1a400000 (size = 0x5800000)
> > EAL: Ask a virtual area of 0x3b200000 bytes
> > EAL: Virtual area found at 0x7faddf000000 (size = 0x3b200000)
> > EAL: Ask a virtual area of 0x400000 bytes
> > EAL: Virtual area found at 0x7faddea00000 (size = 0x400000)
> > EAL: Ask a virtual area of 0x7c00000 bytes
> > EAL: Virtual area found at 0x7fadd6c00000 (size = 0x7c00000)
> > EAL: Ask a virtual area of 0x200000 bytes
> > EAL: Virtual area found at 0x7fadd6800000 (size = 0x200000)
> > EAL: Ask a virtual area of 0x200000 bytes
> > EAL: Virtual area found at 0x7fadd6400000 (size = 0x200000)
> > EAL: Ask a virtual area of 0x200000 bytes
> > EAL: Virtual area found at 0x7fadd6000000 (size = 0x200000)
> > EAL: Ask a virtual area of 0x200000 bytes
> > EAL: Virtual area found at 0x7fadd5c00000 (size = 0x200000)
> > EAL: Ask a virtual area of 0x200000 bytes
> > EAL: Virtual area found at 0x7fadd5800000 (size = 0x200000)
> > EAL: Ask a virtual area of 0x9800000 bytes
> > EAL: Virtual area found at 0x7fadcbe00000 (size = 0x9800000)
> > EAL: Ask a virtual area of 0x19000000 bytes
> > EAL: Virtual area found at 0x7fadb2c00000 (size = 0x19000000)
> > EAL: Ask a virtual area of 0x24400000 bytes
> > EAL: Virtual area found at 0x7fad8e600000 (size = 0x24400000)
> > EAL: Ask a virtual area of 0xc800000 bytes
> > EAL: Virtual area found at 0x7fad81c00000 (size = 0xc800000)
> > EAL: Ask a virtual area of 0x32000000 bytes
> > EAL: Virtual area found at 0x7fad4fa00000 (size = 0x32000000)
> > EAL: Ask a virtual area of 0x3db800000 bytes
> > EAL: Virtual area found at 0x7fa974000000 (size = 0x3db800000)
> > EAL: Ask a virtual area of 0xfa000000 bytes
> > EAL: Virtual area found at 0x7fa879e00000 (size = 0xfa000000)
> > EAL: Ask a virtual area of 0x168400000 bytes
> > EAL: Virtual area found at 0x7fa711800000 (size = 0x168400000)
> > EAL: Ask a virtual area of 0x7d000000 bytes
> > EAL: Virtual area found at 0x7fa694600000 (size = 0x7d000000)
> > EAL: Ask a virtual area of 0x1f4000000 bytes
> > EAL: Virtual area found at 0x7fa4a0400000 (size = 0x1f4000000)
> > EAL: Ask a virtual area of 0x2e400000 bytes
> > EAL: Virtual area found at 0x7fa471e00000 (size = 0x2e400000)
> > EAL: Ask a virtual area of 0x200000 bytes
> > EAL: Virtual area found at 0x7fa471a00000 (size = 0x200000)
> > EAL: Ask a virtual area of 0x200000 bytes
> > EAL: Virtual area found at 0x7fa471600000 (size = 0x200000)
> > EAL: Ask a virtual area of 0x200000 bytes
> > EAL: Virtual area found at 0x7fa471200000 (size = 0x200000)
> > EAL: Ask a virtual area of 0x200000 bytes
> > EAL: Virtual area found at 0x7fa470e00000 (size = 0x200000)
> > EAL: Ask a virtual area of 0x200000 bytes
> > EAL: Virtual area found at 0x7fa470a00000 (size = 0x200000)
> > EAL: Ask a virtual area of 0x200000 bytes
> > EAL: Virtual area found at 0x7fa470600000 (size = 0x200000)
> > EAL: Ask a virtual area of 0x200000 bytes
> > EAL: Virtual area found at 0x7fa470200000 (size = 0x200000)
> > EAL: Ask a virtual area of 0x200000 bytes
> > EAL: Virtual area found at 0x7fa46fe00000 (size = 0x200000)
> > EAL: Ask a virtual area of 0x200000 bytes
> > EAL: Virtual area found at 0x7fa46fa00000 (size = 0x200000)
> > EAL: Ask a virtual area of 0x400000 bytes
> > EAL: Virtual area found at 0x7fa46f400000 (size = 0x400000)
> > EAL: Ask a virtual area of 0x200000 bytes
> > EAL: Virtual area found at 0x7fa46f000000 (size = 0x200000)
> > EAL: Ask a virtual area of 0x400000 bytes
> > EAL: Virtual area found at 0x7fa46ea00000 (size = 0x400000)
> > EAL: Ask a virtual area of 0x200000 bytes
> > EAL: Virtual area found at 0x7fa46e600000 (size = 0x200000)
> > EAL: Ask a virtual area of 0x200000 bytes
> > EAL: Virtual area found at 0x7fa46e200000 (size = 0x200000)
> > EAL: Ask a virtual area of 0x200000 bytes
> > EAL: Virtual area found at 0x7fa46de00000 (size = 0x200000)
> > EAL: Ask a virtual area of 0x200000 bytes
> > EAL: Virtual area found at 0x7fa46da00000 (size = 0x200000)
> > EAL: Ask a virtual area of 0x200000 bytes
> > EAL: Virtual area found at 0x7fa46d600000 (size = 0x200000)
> > EAL: Ask a virtual area of 0x400000 bytes
> > EAL: Virtual area found at 0x7fa46d000000 (size = 0x400000)
> > EAL: Ask a virtual area of 0x200000 bytes
> > EAL: Virtual area found at 0x7fa46cc00000 (size = 0x200000)
> > EAL: Ask a virtual area of 0x200000 bytes
> > EAL: Virtual area found at 0x7fa46c800000 (size = 0x200000)
> > EAL: Ask a virtual area of 0x200000 bytes
> > EAL: Virtual area found at 0x7fa46c400000 (size = 0x200000)
> > EAL: Ask a virtual area of 0x200000 bytes
> > EAL: Virtual area found at 0x7fa46c000000 (size = 0x200000)
> > EAL: Ask a virtual area of 0x200000 bytes
> > EAL: Virtual area found at 0x7fa46bc00000 (size = 0x200000)
> > EAL: Ask a virtual area of 0x200000 bytes
> > EAL: Virtual area found at 0x7fa46b800000 (size = 0x200000)
> > EAL: Ask a virtual area of 0x200000 bytes
> > EAL: Virtual area found at 0x7fa46b400000 (size = 0x200000)
> > EAL: Ask a virtual area of 0x200000 bytes
> > EAL: Virtual area found at 0x7fa46b000000 (size = 0x200000)
> > EAL: Ask a virtual area of 0x400000 bytes
> > EAL: Virtual area found at 0x7fa46aa00000 (size = 0x400000)
> > EAL: Ask a virtual area of 0x200000 bytes
> > EAL: Virtual area found at 0x7fa46a600000 (size = 0x200000)
> > EAL: Ask a virtual area of 0x200000 bytes
> > EAL: Virtual area found at 0x7fa46a200000 (size = 0x200000)
> > EAL: Ask a virtual area of 0x200000 bytes
> > EAL: Virtual area found at 0x7fa469e00000 (size = 0x200000)
> > EAL: Ask a virtual area of 0x200000 bytes
> > EAL: Virtual area found at 0x7fa469a00000 (size = 0x200000)
> > EAL: Ask a virtual area of 0x200000 bytes
> > EAL: Virtual area found at 0x7fa469600000 (size = 0x200000)
> > EAL: Ask a virtual area of 0x200000 bytes
> > EAL: Virtual area found at 0x7fa469200000 (size = 0x200000)
> > EAL: Ask a virtual area of 0x200000 bytes
> > EAL: Virtual area found at 0x7fa468e00000 (size = 0x200000)
> > EAL: Ask a virtual area of 0x200000 bytes
> > EAL: Virtual area found at 0x7fa468a00000 (size = 0x200000)
> > EAL: Ask a virtual area of 0x200000 bytes
> > EAL: Virtual area found at 0x7fa468600000 (size = 0x200000)
> > EAL: Ask a virtual area of 0x200000 bytes
> > EAL: Virtual area found at 0x7fa468200000 (size = 0x200000)
> > EAL: Ask a virtual area of 0x200000 bytes
> > EAL: Virtual area found at 0x7fa467e00000 (size = 0x200000)
> > EAL: Ask a virtual area of 0x400000 bytes
> > EAL: Virtual area found at 0x7fa467800000 (size = 0x400000)
> > EAL: Ask a virtual area of 0x200000 bytes
> > EAL: Virtual area found at 0x7fa467400000 (size = 0x200000)
> > EAL: Ask a virtual area of 0x200000 bytes
> > EAL: Virtual area found at 0x7fa467000000 (size = 0x200000)
> > EAL: Ask a virtual area of 0x200000 bytes
> > EAL: Virtual area found at 0x7fa466c00000 (size = 0x200000)
> > EAL: Ask a virtual area of 0x400000 bytes
> > EAL: Virtual area found at 0x7fa466600000 (size = 0x400000)
> > EAL: Ask a virtual area of 0x200000 bytes
> > EAL: Virtual area found at 0x7fa466200000 (size = 0x200000)
> > EAL: Ask a virtual area of 0x200000 bytes
> > EAL: Virtual area found at 0x7fa465e00000 (size = 0x200000)
> > EAL: Ask a virtual area of 0x200000 bytes
> > EAL: Virtual area found at 0x7fa465a00000 (size = 0x200000)
> > EAL: Ask a virtual area of 0x200000 bytes
> > EAL: Virtual area found at 0x7fa465600000 (size = 0x200000)
> > EAL: Ask a virtual area of 0x400000 bytes
> > EAL: Virtual area found at 0x7fa465000000 (size = 0x400000)
> > EAL: Ask a virtual area of 0x400000 bytes
> > EAL: Virtual area found at 0x7fa464a00000 (size = 0x400000)
> > EAL: Ask a virtual area of 0x200000 bytes
> > EAL: Virtual area found at 0x7fa464600000 (size = 0x200000)
> > EAL: Ask a virtual area of 0x200000 bytes
> > EAL: Virtual area found at 0x7fa464200000 (size = 0x200000)
> > EAL: Ask a virtual area of 0x200000 bytes
> > EAL: Virtual area found at 0x7fa463e00000 (size = 0x200000)
> > EAL: Ask a virtual area of 0x200000 bytes
> > EAL: Virtual area found at 0x7fa463a00000 (size = 0x200000)
> > EAL: Ask a virtual area of 0x200000 bytes
> > EAL: Virtual area found at 0x7fa463600000 (size = 0x200000)
> > EAL: Ask a virtual area of 0x200000 bytes
> > EAL: Virtual area found at 0x7fa463200000 (size = 0x200000)
> > EAL: Ask a virtual area of 0x200000 bytes
> > EAL: Virtual area found at 0x7fa462e00000 (size = 0x200000)
> > EAL: Ask a virtual area of 0x200000 bytes
> > EAL: Virtual area found at 0x7fa462a00000 (size = 0x200000)
> > EAL: Ask a virtual area of 0x200000 bytes
> > EAL: Virtual area found at 0x7fa462600000 (size = 0x200000)
> > EAL: Ask a virtual area of 0x200000 bytes
> > EAL: Virtual area found at 0x7fa462200000 (size = 0x200000)
> > EAL: Ask a virtual area of 0x400000 bytes
> > EAL: Virtual area found at 0x7fa461c00000 (size = 0x400000)
> > EAL: Ask a virtual area of 0x200000 bytes
> > EAL: Virtual area found at 0x7fa461800000 (size = 0x200000)
> > EAL: Ask a virtual area of 0x200000 bytes
> > EAL: Virtual area found at 0x7fa461400000 (size = 0x200000)
> > EAL: Ask a virtual area of 0x200000 bytes
> > EAL: Virtual area found at 0x7fa461000000 (size = 0x200000)
> > EAL: Ask a virtual area of 0x200000 bytes
> > EAL: Virtual area found at 0x7fa460c00000 (size = 0x200000)
> > EAL: Ask a virtual area of 0x200000 bytes
> > EAL: Virtual area found at 0x7fa460800000 (size = 0x200000)
> > EAL: Ask a virtual area of 0x200000 bytes
> > EAL: Virtual area found at 0x7fa460400000 (size = 0x200000)
> > EAL: Ask a virtual area of 0x200000 bytes
> > EAL: Virtual area found at 0x7fa460000000 (size = 0x200000)
> > EAL: Ask a virtual area of 0x200000 bytes
> > EAL: Virtual area found at 0x7fa45fc00000 (size = 0x200000)
> > EAL: Ask a virtual area of 0x200000 bytes
> > EAL: Virtual area found at 0x7fa45f800000 (size = 0x200000)
> > EAL: Ask a virtual area of 0x400000 bytes
> > EAL: Virtual area found at 0x7fa45f200000 (size = 0x400000)
> > EAL: Ask a virtual area of 0x200000 bytes
> > EAL: Virtual area found at 0x7fa45ee00000 (size = 0x200000)
> > EAL: Ask a virtual area of 0x200000 bytes
> > EAL: Virtual area found at 0x7fa45ea00000 (size = 0x200000)
> > EAL: Ask a virtual area of 0x200000 bytes
> > EAL: Virtual area found at 0x7fa45e600000 (size = 0x200000)
> > EAL: Ask a virtual area of 0x200000 bytes
> > EAL: Virtual area found at 0x7fa45e200000 (size = 0x200000)
> > EAL: Ask a virtual area of 0x200000 bytes
> > EAL: Virtual area found at 0x7fa45de00000 (size = 0x200000)
> > EAL: Ask a virtual area of 0x200000 bytes
> > EAL: Virtual area found at 0x7fa45da00000 (size = 0x200000)
> > EAL: Ask a virtual area of 0x200000 bytes
> > EAL: Virtual area found at 0x7fa45d600000 (size = 0x200000)
> > EAL: Ask a virtual area of 0x200000 bytes
> > EAL: Virtual area found at 0x7fa45d200000 (size = 0x200000)
> > EAL: Ask a virtual area of 0x200000 bytes
> > EAL: Virtual area found at 0x7fa45ce00000 (size = 0x200000)
> > EAL: Ask a virtual area of 0x200000 bytes
> > EAL: Virtual area found at 0x7fa45ca00000 (size = 0x200000)
> > EAL: Ask a virtual area of 0x200000 bytes
> > EAL: Virtual area found at 0x7fa45c600000 (size = 0x200000)
> > EAL: Ask a virtual area of 0x200000 bytes
> > EAL: Virtual area found at 0x7fa45c200000 (size = 0x200000)
> > EAL: Ask a virtual area of 0x200000 bytes
> > EAL: Virtual area found at 0x7fa45be00000 (size = 0x200000)
> > EAL: Ask a virtual area of 0x200000 bytes
> > EAL: Virtual area found at 0x7fa45ba00000 (size = 0x200000)
> > EAL: Ask a virtual area of 0x200000 bytes
> > EAL: Virtual area found at 0x7fa45b600000 (size = 0x200000)
> > EAL: Ask a virtual area of 0x400000 bytes
> > EAL: Virtual area found at 0x7fa45b000000 (size = 0x400000)
> > EAL: Ask a virtual area of 0x200000 bytes
> > EAL: Virtual area found at 0x7fa45ac00000 (size = 0x200000)
> > EAL: Ask a virtual area of 0x200000 bytes
> > EAL: Virtual area found at 0x7fa45a800000 (size = 0x200000)
> > EAL: Ask a virtual area of 0x400000 bytes
> > EAL: Virtual area found at 0x7fa45a200000 (size = 0x400000)
> > EAL: Ask a virtual area of 0x200000 bytes
> > EAL: Virtual area found at 0x7fa459e00000 (size = 0x200000)
> > EAL: Ask a virtual area of 0x200000 bytes
> > EAL: Virtual area found at 0x7fa459a00000 (size = 0x200000)
> > EAL: Ask a virtual area of 0x200000 bytes
> > EAL: Virtual area found at 0x7fa459600000 (size = 0x200000)
> > EAL: Ask a virtual area of 0x200000 bytes
> > EAL: Virtual area found at 0x7fa459200000 (size = 0x200000)
> > EAL: Ask a virtual area of 0x400000 bytes
> > EAL: Virtual area found at 0x7fa458c00000 (size = 0x400000)
> > EAL: Ask a virtual area of 0x400000 bytes
> > EAL: Virtual area found at 0x7fa458600000 (size = 0x400000)
> > EAL: Ask a virtual area of 0x200000 bytes
> > EAL: Virtual area found at 0x7fa458200000 (size = 0x200000)
> > EAL: Ask a virtual area of 0x200000 bytes
> > EAL: Virtual area found at 0x7fa457e00000 (size = 0x200000)
> > EAL: Ask a virtual area of 0x200000 bytes
> > EAL: Virtual area found at 0x7fa457a00000 (size = 0x200000)
> > EAL: Ask a virtual area of 0x200000 bytes
> > EAL: Virtual area found at 0x7fa457600000 (size = 0x200000)
> > EAL: Ask a virtual area of 0x200000 bytes
> > EAL: Virtual area found at 0x7fa457200000 (size = 0x200000)
> > EAL: Ask a virtual area of 0x200000 bytes
> > EAL: Virtual area found at 0x7fa456e00000 (size = 0x200000)
> > EAL: Ask a virtual area of 0x200000 bytes
> > EAL: Virtual area found at 0x7fa456a00000 (size = 0x200000)
> > EAL: Ask a virtual area of 0x200000 bytes
> > EAL: Virtual area found at 0x7fa456600000 (size = 0x200000)
> > EAL: Ask a virtual area of 0x139400000 bytes
> > EAL: Virtual area found at 0x7fa31d000000 (size = 0x139400000)
> > EAL: Ask a virtual area of 0xe9000000 bytes
> > EAL: Virtual area found at 0x7fa233e00000 (size = 0xe9000000)
> > EAL: Ask a virtual area of 0x188000000 bytes
> > EAL: Virtual area found at 0x7fa0abc00000 (size = 0x188000000)
> > EAL: map_all_hugepages(): mmap failed: Cannot allocate memory
> > EAL: Failed to remap 2 MB pages
> > PANIC in rte_eal_init():
> > Cannot init memory
> > Aborted (core dumped)
> >
> >
> >
> >
> > .
> >
> 
> --
> Regards,
> Haifeng



More information about the dev mailing list