[RFC PATCH v2] mempool: optimizations

[Bruce Richardson]

On Wed, Aug 12, 2026 at 12:06:26PM +0000, Morten Brørup wrote:
> This patch introduces some mempool optimizations, which might be
> controversial.
> 
> 1. Access local cache without first accessing the mempool header struct.
> 
> When getting/putting objects in a mempool cache, it required accessing
> the "size" field and "local_cache" pointer in the mempool structure, to
> determine if the local cache was present.
> 
> The mempool structure was changed, so the local cache array is now an
> integral part of the mempool structure.  This means that local cache can
> be accessed directly, without first checking the "size" and "local_cache"
> fields in the mempool header structure.  This avoids a couple of load
> operations with a potential CPU cache miss when the mempool header itself
> is not hot in the CPU cache.  The "local_cache" field was changed from
> being a pointer to the local cache array, and instead became the local
> cache array itself.
> 
> This change similarly speeds up rte_mempool_get_priv(), because it no
> longer needs to access the mempool header structure (specifically, the
> "cache_size" field) to determine the address of the mempool's private
> data.
> 
> Disadvantage: Memory for local cache is also consumed by mempools
> configured without cache.
> 
> Related changes: - The mempool cache audit function was improved.  - The
> mempool autotest accessed the internal RTE_MEMPOOL_HEADER_SIZE macro, and
> was updated accordingly.
> 
> 2. Move objects in mempool cache as 32-byte chunks at CPU cache line
> aligned addresses.
> 
> Improved memory copy performance by ensuring that objects in mempool
> cache can be moved as 32-byte chunks at CPU cache line aligned addresses.
> 
> This introduces a new requirement: The mempool cache size must be
> divisible by 32.  The new requirement may be beneficial for future
> purposes.
> 
> Disadvantage: Mempool cache size must be divisible by 32.  For
> compatibility purposes, a requested cache size not divisible by 32 is
> handled by a graceful fallback at mempool cache creation and mempool
> creation.
> 
> Related changes: - The TAP driver used a mempool cache size of 4 mbufs
> for GSO, and was updated to a cache size of 32 mbufs.
> 
> Other changes: - The description of the RTE_MEMPOOL_NAMESIZE macro was
> expanded to explain how the value is derived.
> 
> Signed-off-by: Morten Brørup <mb at smartsharesystems.com> --- v2: * Removed
> patch dependency, and included dependent patch instead.  Trying to fix
> apply patch failure.  v2 resend: * Improved patch description.  ---
> app/test/test_mempool.c                 |  3 +-
> doc/guides/rel_notes/release_26_11.rst  |  6 ++
> drivers/net/sxe2/sxe2_txrx_vec_avx512.c |  2 +-
> drivers/net/tap/rte_eth_tap.c           |  2 +-
> lib/eal/include/rte_common.h            | 12 +++
> lib/mempool/mempool_trace.h             |  1 - lib/mempool/rte_mempool.c
> | 76 +++++++++++++------ lib/mempool/rte_mempool.h               | 97
> ++++++++++++++----------- 8 files changed, 131 insertions(+), 68
> deletions(-)
> 
> diff --git a/app/test/test_mempool.c b/app/test/test_mempool.c index
> e54249ce61..76d45cea2a 100644 --- a/app/test/test_mempool.c +++
> b/app/test/test_mempool.c @@ -112,8 +112,7 @@ test_mempool_basic(struct
> rte_mempool *mp, int use_external_cache) GOTO_ERR(ret, out);
>  
>  	printf("get private data\n"); -	if (rte_mempool_get_priv(mp) !=
>  	(char *)mp + -			RTE_MEMPOOL_HEADER_SIZE(mp,
>  	mp->cache_size)) +	if (rte_mempool_get_priv(mp) != (char *)mp
>  	+ sizeof(struct rte_mempool)) GOTO_ERR(ret, out);
>  
>  #ifndef RTE_EXEC_ENV_FREEBSD /* rte_mem_virt2iova() not supported on bsd
>  */ diff --git a/doc/guides/rel_notes/release_26_11.rst
>  b/doc/guides/rel_notes/release_26_11.rst index c8cc86295d..e31325585a
>  100644 --- a/doc/guides/rel_notes/release_26_11.rst +++
>  b/doc/guides/rel_notes/release_26_11.rst @@ -68,6 +68,8 @@ Removed Items
>  Also, make sure to start the actual text at the margin.
>  =======================================================
>  
> +* mempool: The obsolete ``flushthresh`` field was removed from the
> ``rte_mempool_cache`` structure.  + * Removed deprecated symbols:
>  
I'm not convinced about removing this field at this point. Based on
previous discussions around run-to-completion vs pipeline apps, and the
reported performance degradations due to recent cache changes, I could
see a scenario where it's useful to track a separate flushthreshold or
cache-keep threshold for a mempool.

/Bruce