[dpdk-users] Optimal number of elements in mempool n = (2^q - 1) vs examples, what is the right thing to do?

[Pavel Shirshov]

Hi Olivier,

I got your point. Thank you for explaining.
My point is about following. DPDK is not so traditional library. This
library is used for high performance applications, when every detail
is important.
Your initial answer with details how mempool is implemented showed
some DPDK internal details. Which was very valuable for me.
I'm reading dpdk-users to get this kind of internal information about
DPDK, because DPDK has a good documentation, but not comprehensive
one. dpdk-users gives much more valuable information, than DPDK
documentation itself.

Thanks

On Tue, Feb 7, 2017 at 12:53 AM, Olivier MATZ <olivier.matz at 6wind.com> wrote:
> Hi Pavel,
>
> On Mon, 6 Feb 2017 13:07:35 -0800, Pavel Shirshov
> <pavel.shirshov at gmail.com> wrote:
>> On Mon, Feb 6, 2017 at 5:51 AM, Olivier Matz <olivier.matz at 6wind.com>
>> wrote:
>> > Hi Vlad,
>> >
>> > On Wed, 1 Feb 2017 16:54:57 +0000, Vlad.Lazarenko at worldquant.com
>> > (Lazarenko, Vlad (WorldQuant)) wrote:
>> >> Hello,
>> >>
>> >> I'm new to DPDK and have noticed that documentation for
>> >> rte_mempool_create states that the optimal size for a number of
>> >> elements in the pool is n = (2^q-1). But in many examples it is
>> >> simply set to 2^q (multi_process/simple_mp/main.c uses 2014, for
>> >> example). This is a bit confusing. Is 2^q - 1 really the optimal
>> >> number but examples don't use it, or maybe the documentation for
>> >> the mempool is wrong, or...? If anyone could shed some light on
>> >> this that'd be helpful.
>> >
>> > That's true for rte_mempool based on a rte_ring (this is the
>> > default, but since recently, it's possible to use another handler).
>> >
>> > The size of a rte_ring is (2^n - 1), because one element in the
>> > ring is reserved to distinguish between a full an an empty ring.
>> > So, when a mempool uses a ring, if we ask for 2^n elements, a ring
>> > of size (2^(n+1) - 1) is created, which can consume additional
>> > memory.
>> >
>> > On the other hand, the mempool object size is often much larger than
>> > a ring entry (usually 8 bytes, the size of a pointer), especially
>> > knowing that by default, the objects are cache aligned (usually 64
>> > bytes).
>> >
>> > So we may remove this note in the future since it's not very
>> > relevant.
>>
>> Hi Olivier,
>>
>> It's a good explanation of rte_mempool internals. I think it would be
>> good to have your comment in the rte_mempool documentation. Could we
>> add it there?
>
> My comment applies on mempool based on rings (default), but it becomes
> wrong when using another handler (ex: stack, or upcoming hw handlers).
> Moreover, the size of the pointer array in ring is often negligible in
> comparison with the size of the objects array in mempool, making this
> comment no so useful.
>
> So I'll probably remove this part from API guide to the programmer's
> guide.
>
> Regards,
> Olivier