[dpdk-dev] [PATCH] mempool: fix slow allocation of large mempools

[Burakov, Anatoly]

On 09-Jan-20 1:27 PM, Olivier Matz wrote:
> When allocating a mempool which is larger than the largest
> available area, it can take a lot of time:
> 
> a- the mempool calculate the required memory size, and tries
>     to allocate it, it fails
> b- then it tries to allocate the largest available area (this
>     does not request new huge pages)
> c- add this zone to the mempool, this triggers the allocation
>     of a mem hdr, which request a new huge page
> d- back to a- until mempool is populated or until there is no
>     more memory
> 
> This can take a lot of time to finally fail (several minutes): in step
> a- it takes all available hugepages on the system, then release them
> after it fails.
> 
> The problem appeared with commit eba11e364614 ("mempool: reduce wasted
> space on populate"), because smaller chunks are now allowed. Previously,
> it had to be at least one page size, which is not the case in step b-.
> 
> To fix this, implement our own way to allocate the largest available
> area instead of using the feature from memzone: if an allocation fails,
> try to divide the size by 2 and retry. When the requested size falls
> below min_chunk_size, stop and return an error.
> 
> Fixes: eba11e364614 ("mempool: reduce wasted space on populate")
> Cc: stable at dpdk.org
> 
> Signed-off-by: Olivier Matz <olivier.matz at 6wind.com>
> ---

I don't particularly like the idea of working around this issue as 
opposed to fixing it memzone-side, but since there's currently no plan 
to address this in memzone allocator, this should work much better than 
before.

Acked-by: Anatoly Burakov <anatoly.burakov at intel.com>

-- 
Thanks,
Anatoly