[dpdk-dev] [PATCH v3] eal: add new prefetch write variants

[Harry van Haaren]

This commit adds new rte_prefetchX_write() variants, that suggest to the
compiler to use a prefetch instruction with intention to write. As a
compiler builtin, the compiler can choose based on compilation target
what the best implementation for this instruction is.

Three versions are provided, targeting the different levels of cache.

Signed-off-by: Harry van Haaren <harry.van.haaren at intel.com>
Reviewed-by: Jerin Jacob <jerinj at marvell.com>
Reviewed-by: Ruifeng Wang <ruifeng.wang at arm.com>

---

v3:
- Add reviewed by tags from Jerin and Ruifeng, thanks!
- Add __rte_experimental as they are new functions (David)
  This required adding the rte_compat.h include.
- Rework return type to new line (Jerin)
- Add calls in test_prefetch.c to new functions (David)
- Add item to release notes (David)

v2:
- Add L1, L2, and L3 variants as ARM64 uarch supports them (Pavan)

The integer constants passed to the builtin are not available as
a #define value, and doing #defines just for this write variant
does not seems a nice solution to me... particularly for those using
IDEs where any #define value is auto-hinted for code-completion.
---
 app/test/test_prefetch.c                      |  4 ++
 doc/guides/rel_notes/release_20_11.rst        |  6 ++
 lib/librte_eal/include/generic/rte_prefetch.h | 57 +++++++++++++++++++
 3 files changed, 67 insertions(+)

diff --git a/app/test/test_prefetch.c b/app/test/test_prefetch.c
index 41f219af78..32e08f8afe 100644
--- a/app/test/test_prefetch.c
+++ b/app/test/test_prefetch.c
@@ -26,6 +26,10 @@ test_prefetch(void)
 	rte_prefetch1(&a);
 	rte_prefetch2(&a);
 
+	rte_prefetch0_write(&a);
+	rte_prefetch1_write(&a);
+	rte_prefetch2_write(&a);
+
 	return 0;
 }
 
diff --git a/doc/guides/rel_notes/release_20_11.rst b/doc/guides/rel_notes/release_20_11.rst
index 0925123e9c..8b51ef0dbc 100644
--- a/doc/guides/rel_notes/release_20_11.rst
+++ b/doc/guides/rel_notes/release_20_11.rst
@@ -62,6 +62,12 @@ New Features
   The functions are provided as a generic stubs and
   x86 specific implementation.
 
+* **Added prefetch with intention to write APIs.**
+
+  Added new prefetch function variants e.g. ``rte_prefetch0_write``,
+  which allow the programmer to prefetch a cache line and also indicate
+  the intention to write.
+
 * **Updated CRC modules of the net library.**
 
   * Added runtime selection of the optimal architecture-specific CRC path.
diff --git a/lib/librte_eal/include/generic/rte_prefetch.h b/lib/librte_eal/include/generic/rte_prefetch.h
index 6e47bdfbad..53d68c40f1 100644
--- a/lib/librte_eal/include/generic/rte_prefetch.h
+++ b/lib/librte_eal/include/generic/rte_prefetch.h
@@ -5,6 +5,8 @@
 #ifndef _RTE_PREFETCH_H_
 #define _RTE_PREFETCH_H_
 
+#include "rte_compat.h"
+
 /**
  * @file
  *
@@ -51,4 +53,59 @@ static inline void rte_prefetch2(const volatile void *p);
  */
 static inline void rte_prefetch_non_temporal(const volatile void *p);
 
+/**
+ * Prefetch a cache line into all cache levels, with intention to write. This
+ * prefetch variant hints to the CPU that the program is expecting to write to
+ * the cache line being prefetched.
+ *
+ * @param p Address to prefetch
+ */
+__rte_experimental
+static inline void
+rte_prefetch0_write(const void *p)
+{
+	/* 1 indicates intention to write, 3 sets target cache level to L1. See
+	 * GCC docs where these integer constants are described in more detail:
+	 *  https://gcc.gnu.org/onlinedocs/gcc/Other-Builtins.html
+	 */
+	__builtin_prefetch(p, 1, 3);
+}
+
+/**
+ * Prefetch a cache line into all cache levels, except the 0th, with intention
+ * to write. This prefetch variant hints to the CPU that the program is
+ * expecting to write to the cache line being prefetched.
+ *
+ * @param p Address to prefetch
+ */
+__rte_experimental
+static inline void
+rte_prefetch1_write(const void *p)
+{
+	/* 1 indicates intention to write, 2 sets target cache level to L2. See
+	 * GCC docs where these integer constants are described in more detail:
+	 *  https://gcc.gnu.org/onlinedocs/gcc/Other-Builtins.html
+	 */
+	__builtin_prefetch(p, 1, 2);
+}
+
+/**
+ * Prefetch a cache line into all cache levels, except the 0th and 1st, with
+ * intention to write. This prefetch variant hints to the CPU that the program
+ * is expecting to write to the cache line being prefetched.
+ *
+ * @param p Address to prefetch
+ */
+__rte_experimental
+static inline void
+rte_prefetch2_write(const void *p)
+{
+	/* 1 indicates intention to write, 1 sets target cache level to L3. See
+	 * GCC docs where these integer constants are described in more detail:
+	 *  https://gcc.gnu.org/onlinedocs/gcc/Other-Builtins.html
+	 */
+	__builtin_prefetch(p, 1, 1);
+}
+
+
 #endif /* _RTE_PREFETCH_H_ */
-- 
2.17.1