[dpdk-dev] [PATCH v9 1/2] mem: balanced allocation of hugepages

Sergio Gonzalez Monroy sergio.gonzalez.monroy at intel.com
Wed Jun 28 12:30:31 CEST 2017


On 27/06/2017 11:24, Ilya Maximets wrote:
> Currently EAL allocates hugepages one by one not paying attention
> from which NUMA node allocation was done.
>
> Such behaviour leads to allocation failure if number of available
> hugepages for application limited by cgroups or hugetlbfs and
> memory requested not only from the first socket.
>
> Example:
> 	# 90 x 1GB hugepages availavle in a system
>
> 	cgcreate -g hugetlb:/test
> 	# Limit to 32GB of hugepages
> 	cgset -r hugetlb.1GB.limit_in_bytes=34359738368 test
> 	# Request 4GB from each of 2 sockets
> 	cgexec -g hugetlb:test testpmd --socket-mem=4096,4096 ...
>
> 	EAL: SIGBUS: Cannot mmap more hugepages of size 1024 MB
> 	EAL: 32 not 90 hugepages of size 1024 MB allocated
> 	EAL: Not enough memory available on socket 1!
> 	     Requested: 4096MB, available: 0MB
> 	PANIC in rte_eal_init():
> 	Cannot init memory
>
> 	This happens beacause all allocated pages are
> 	on socket 0.
>
> Fix this issue by setting mempolicy MPOL_PREFERRED for each hugepage
> to one of requested nodes using following schema:
>
> 	1) Allocate essential hugepages:
> 		1.1) Allocate as many hugepages from numa N to
> 		     only fit requested memory for this numa.
> 		1.2) repeat 1.1 for all numa nodes.
> 	2) Try to map all remaining free hugepages in a round-robin
> 	   fashion.
> 	3) Sort pages and choose the most suitable.
>
> In this case all essential memory will be allocated and all remaining
> pages will be fairly distributed between all requested nodes.
>
> New config option RTE_EAL_NUMA_AWARE_HUGEPAGES introduced and
> enabled by default for linuxapp except armv7 and dpaa2.
> Enabling of this option adds libnuma as a dependency for EAL.
>
> Fixes: 77988fc08dc5 ("mem: fix allocating all free hugepages")
>
> Signed-off-by: Ilya Maximets <i.maximets at samsung.com>
> ---
>   config/common_base                        |   1 +
>   config/common_linuxapp                    |   1 +
>   config/defconfig_arm-armv7a-linuxapp-gcc  |   3 +
>   config/defconfig_arm64-dpaa2-linuxapp-gcc |   3 +
>   lib/librte_eal/linuxapp/eal/Makefile      |   3 +
>   lib/librte_eal/linuxapp/eal/eal_memory.c  | 120 ++++++++++++++++++++++++++++--
>   mk/rte.app.mk                             |   3 +
>   7 files changed, 126 insertions(+), 8 deletions(-)

Good stuff Ilya!

Hemant, Jerin, could you also ack the patch if you are happy with it? 
Thanks.

Acked-by: Sergio Gonzalez Monroy <sergio.gonzalez.monroy at intel.com>



More information about the dev mailing list