memif insufficient padding

[Mattias Rönnblom]

On 2024-08-28 23:04, Morten Brørup wrote:
> Jakub,
> 
> While browsing virtual interfaces in DPDK, I noticed a possible performance issue in the memif driver:
> 
> If "head" and "tail" are accessed by different lcores, they are not sufficiently far away from each other (and other hot fields) to prevent false sharing-like effects on systems with a next-N-lines hardware prefetcher, which will prefetch "tail" when fetching "head", and prefetch "head" when fetching "flags".
> 
> I suggest updating the structure somewhat like this:
> 
> -#define MEMIF_CACHELINE_ALIGN_MARK(mark) \
> -	alignas(RTE_CACHE_LINE_SIZE) RTE_MARKER mark;
> -
> -typedef struct {
> -	MEMIF_CACHELINE_ALIGN_MARK(cacheline0);
> +typedef struct __rte_cache_aligned {
> 	uint32_t cookie;			/**< MEMIF_COOKIE */
> 	uint16_t flags;				/**< flags */
> #define MEMIF_RING_FLAG_MASK_INT 1		/**< disable interrupt mode */
> +	RTE_CACHE_GUARD; /* isolate head from flags */

Wouldn't it be better to cache align the 'head' (or cache-aligned 'head' 
*and* add a RTE_CACHE_GUARD)? In other words, isn't the purpose of 
RTE_CACHE_GUARD to provide zero or more cache line of extra padding, 
rather than a mechanism to avoid same-cache line false sharing?

> 	RTE_ATOMIC(uint16_t) head;			/**< pointer to ring buffer head */
> -	MEMIF_CACHELINE_ALIGN_MARK(cacheline1);
> +	RTE_CACHE_GUARD; /* isolate tail from head */
> 	RTE_ATOMIC(uint16_t) tail;			/**< pointer to ring buffer tail */
> -	MEMIF_CACHELINE_ALIGN_MARK(cacheline2);
> +	RTE_CACHE_GUARD; /* isolate descriptors from tail */
> -	memif_desc_t desc[0];			/**< buffer descriptors */
> +	memif_desc_t desc[];			/**< buffer descriptors */
> } memif_ring_t;
> 
> 
> Med venlig hilsen / Kind regards,
> -Morten Brørup
>