patch 'ring: establish a safe partial order in hts-ring' has been queued to stable release 23.11.6
Shani Peretz
shperetz at nvidia.com
Thu Dec 25 10:18:31 CET 2025
Hi,
FYI, your patch has been queued to stable release 23.11.6
Note it hasn't been pushed to http://dpdk.org/browse/dpdk-stable yet.
It will be pushed if I get no objections before 12/30/25. So please
shout if anyone has objections.
Also note that after the patch there's a diff of the upstream commit vs the
patch applied to the branch. This will indicate if there was any rebasing
needed to apply to the stable branch. If there were code changes for rebasing
(ie: not only metadata diffs), please double check that the rebase was
correctly done.
Queued patches are on a temporary branch at:
https://github.com/shanipr/dpdk-stable
This queued commit can be viewed at:
https://github.com/shanipr/dpdk-stable/commit/60f8d1394655899888f2a17c934ab04d9c62e934
Thanks.
Shani
---
>From 60f8d1394655899888f2a17c934ab04d9c62e934 Mon Sep 17 00:00:00 2001
From: Wathsala Vithanage <wathsala.vithanage at arm.com>
Date: Tue, 2 Dec 2025 20:39:27 +0000
Subject: [PATCH] ring: establish a safe partial order in hts-ring
[ upstream commit 66d5f962780694f6aebf000907fc3ce7a72584f9 ]
Enforce a safe partial order by making the CAS and the preceding head
load use release and acquire semantics. This creates a pairwise
happens-before relationship between threads of the same role.
Combine the two load-acquire operations of ht.raw, which were previously
split across the two paths of a conditional branch, into
__rte_ring_hts_head_wait. This simplifies the branching logic and makes
the synchronization behavior easier to understand.
Add comments to explain synchronizes with edges in detail.
Signed-off-by: Wathsala Vithanage <wathsala.vithanage at arm.com>
Signed-off-by: Ola Liljedahl <ola.liljedahl at arm.com>
---
lib/ring/rte_ring_hts_elem_pvt.h | 96 +++++++++++++++++++++++---------
1 file changed, 71 insertions(+), 25 deletions(-)
diff --git a/lib/ring/rte_ring_hts_elem_pvt.h b/lib/ring/rte_ring_hts_elem_pvt.h
index 91f5eeccb9..3c27197b6b 100644
--- a/lib/ring/rte_ring_hts_elem_pvt.h
+++ b/lib/ring/rte_ring_hts_elem_pvt.h
@@ -32,22 +32,40 @@ __rte_ring_hts_update_tail(struct rte_ring_hts_headtail *ht, uint32_t old_tail,
RTE_SET_USED(enqueue);
tail = old_tail + num;
+
+ /*
+ * R0: Release the tail update. Establishes a synchronization edge with
+ * the load-acquire at A1/A3. This release ensures that all updates to
+ * *ht and the ring array made by this thread become visible to the
+ * opposing thread once the tail value written here is observed.
+ */
rte_atomic_store_explicit(&ht->ht.pos.tail, tail, rte_memory_order_release);
}
/**
- * @internal waits till tail will become equal to head.
- * Means no writer/reader is active for that ring.
- * Suppose to work as serialization point.
+ * @internal
+ * Waits until the tail becomes equal to the head.
+ * This indicates that another thread has finished its transaction, and there
+ * is a chance that we could be the next writer or reader in line.
+ *
+ * Returns ht.raw at this point. The value may be imprecise, since another
+ * thread might change the state before we observe ht.raw, but that does not
+ * matter. The function __rte_ring_hts_move_head() can detect and recall this
+ * function when it reaches the linearization point (CAS).
*/
-static __rte_always_inline void
+static __rte_always_inline union __rte_ring_hts_pos
__rte_ring_hts_head_wait(const struct rte_ring_hts_headtail *ht,
- union __rte_ring_hts_pos *p)
+ int memorder)
{
- while (p->pos.head != p->pos.tail) {
+ union __rte_ring_hts_pos p;
+ p.raw = rte_atomic_load_explicit(&ht->ht.raw, memorder);
+
+ while (p.pos.head != p.pos.tail) {
rte_pause();
- p->raw = rte_atomic_load_explicit(&ht->ht.raw, rte_memory_order_acquire);
+ p.raw = rte_atomic_load_explicit(&ht->ht.raw, memorder);
}
+
+ return p;
}
/**
@@ -58,13 +76,11 @@ __rte_ring_hts_move_prod_head(struct rte_ring *r, unsigned int num,
enum rte_ring_queue_behavior behavior, uint32_t *old_head,
uint32_t *free_entries)
{
- uint32_t n;
+ uint32_t n, cons_tail;
union __rte_ring_hts_pos np, op;
const uint32_t capacity = r->capacity;
- op.raw = rte_atomic_load_explicit(&r->hts_prod.ht.raw, rte_memory_order_acquire);
-
do {
/* Reset n to the initial burst count */
n = num;
@@ -74,7 +90,20 @@ __rte_ring_hts_move_prod_head(struct rte_ring *r, unsigned int num,
* make sure that we read prod head/tail *before*
* reading cons tail.
*/
- __rte_ring_hts_head_wait(&r->hts_prod, &op);
+ /*
+ * A0: Synchronizes with the CAS at R1.
+ * Establishes a happens-before relationship with a thread of the same
+ * type that released the ht.raw, ensuring this thread observes all of
+ * its memory effects needed to maintain a safe partial order.
+ */
+ op = __rte_ring_hts_head_wait(&r->hts_prod, rte_memory_order_acquire);
+
+ /*
+ * A1: Establish a synchronizes-with edge using a store-release at R0.
+ * This ensures that all memory effects from the preceding opposing
+ * thread are observed.
+ */
+ cons_tail = rte_atomic_load_explicit(&r->cons.tail, rte_memory_order_acquire);
/*
* The subtraction is done between two unsigned 32bits value
@@ -82,7 +111,7 @@ __rte_ring_hts_move_prod_head(struct rte_ring *r, unsigned int num,
* *old_head > cons_tail). So 'free_entries' is always between 0
* and capacity (which is < size).
*/
- *free_entries = capacity + r->cons.tail - op.pos.head;
+ *free_entries = capacity + cons_tail - op.pos.head;
/* check that we have enough room in ring */
if (unlikely(n > *free_entries))
@@ -96,13 +125,16 @@ __rte_ring_hts_move_prod_head(struct rte_ring *r, unsigned int num,
np.pos.head = op.pos.head + n;
/*
- * this CAS(ACQUIRE, ACQUIRE) serves as a hoist barrier to prevent:
- * - OOO reads of cons tail value
- * - OOO copy of elems from the ring
+ * R1: Establishes a synchronizes-with edge with the load-acquire
+ * of ht.raw at A0. This makes sure that the store-release to the
+ * tail by this thread, if it was of the opposite type, becomes
+ * visible to another thread of the current type. That thread will
+ * then observe the updates in the same order, keeping a safe
+ * partial order.
*/
} while (rte_atomic_compare_exchange_strong_explicit(&r->hts_prod.ht.raw,
(uint64_t *)(uintptr_t)&op.raw, np.raw,
- rte_memory_order_acquire, rte_memory_order_acquire) == 0);
+ rte_memory_order_release, rte_memory_order_relaxed) == 0);
*old_head = op.pos.head;
return n;
@@ -116,11 +148,9 @@ __rte_ring_hts_move_cons_head(struct rte_ring *r, unsigned int num,
enum rte_ring_queue_behavior behavior, uint32_t *old_head,
uint32_t *entries)
{
- uint32_t n;
+ uint32_t n, prod_tail;
union __rte_ring_hts_pos np, op;
- op.raw = rte_atomic_load_explicit(&r->hts_cons.ht.raw, rte_memory_order_acquire);
-
/* move cons.head atomically */
do {
/* Restore n as it may change every loop */
@@ -131,14 +161,27 @@ __rte_ring_hts_move_cons_head(struct rte_ring *r, unsigned int num,
* make sure that we read cons head/tail *before*
* reading prod tail.
*/
- __rte_ring_hts_head_wait(&r->hts_cons, &op);
+ /*
+ * A2: Synchronizes with the CAS at R2.
+ * Establishes a happens-before relationship with a thread of the same
+ * type that released the ht.raw, ensuring this thread observes all of
+ * its memory effects needed to maintain a safe partial order.
+ */
+ op = __rte_ring_hts_head_wait(&r->hts_cons, rte_memory_order_acquire);
+
+ /*
+ * A3: Establish a synchronizes-with edge using a store-release at R0.
+ * This ensures that all memory effects from the preceding opposing
+ * thread are observed.
+ */
+ prod_tail = rte_atomic_load_explicit(&r->prod.tail, rte_memory_order_acquire);
/* The subtraction is done between two unsigned 32bits value
* (the result is always modulo 32 bits even if we have
* cons_head > prod_tail). So 'entries' is always between 0
* and size(ring)-1.
*/
- *entries = r->prod.tail - op.pos.head;
+ *entries = prod_tail - op.pos.head;
/* Set the actual entries for dequeue */
if (n > *entries)
@@ -151,13 +194,16 @@ __rte_ring_hts_move_cons_head(struct rte_ring *r, unsigned int num,
np.pos.head = op.pos.head + n;
/*
- * this CAS(ACQUIRE, ACQUIRE) serves as a hoist barrier to prevent:
- * - OOO reads of prod tail value
- * - OOO copy of elems from the ring
+ * R2: Establishes a synchronizes-with edge with the load-acquire
+ * of ht.raw at A2. This makes sure that the store-release to the
+ * tail by this thread, if it was of the opposite type, becomes
+ * visible to another thread of the current type. That thread will
+ * then observe the updates in the same order, keeping a safe
+ * partial order.
*/
} while (rte_atomic_compare_exchange_strong_explicit(&r->hts_cons.ht.raw,
(uint64_t *)(uintptr_t)&op.raw, np.raw,
- rte_memory_order_acquire, rte_memory_order_acquire) == 0);
+ rte_memory_order_release, rte_memory_order_relaxed) == 0);
*old_head = op.pos.head;
return n;
--
2.43.0
---
Diff of the applied patch vs upstream commit (please double-check if non-empty:
---
--- - 2025-12-25 11:16:39.845569579 +0200
+++ 0070-ring-establish-a-safe-partial-order-in-hts-ring.patch 2025-12-25 11:16:36.073824000 +0200
@@ -1 +1 @@
-From 66d5f962780694f6aebf000907fc3ce7a72584f9 Mon Sep 17 00:00:00 2001
+From 60f8d1394655899888f2a17c934ab04d9c62e934 Mon Sep 17 00:00:00 2001
@@ -3,2 +3,4 @@
-Date: Tue, 11 Nov 2025 18:37:18 +0000
-Subject: [PATCH] ring: establish safe partial order in HTS mode
+Date: Tue, 2 Dec 2025 20:39:27 +0000
+Subject: [PATCH] ring: establish a safe partial order in hts-ring
+
+[ upstream commit 66d5f962780694f6aebf000907fc3ce7a72584f9 ]
@@ -17,3 +18,0 @@
-Fixes: 1cc363b8ce06e ("ring: introduce HTS ring mode")
-Cc: stable at dpdk.org
-
@@ -23,2 +22,2 @@
- lib/ring/rte_ring_hts_elem_pvt.h | 66 ++++++++++++++++++++++++--------
- 1 file changed, 49 insertions(+), 17 deletions(-)
+ lib/ring/rte_ring_hts_elem_pvt.h | 96 +++++++++++++++++++++++---------
+ 1 file changed, 71 insertions(+), 25 deletions(-)
@@ -27 +26 @@
-index e2b82dd1e6..a01089d15d 100644
+index 91f5eeccb9..3c27197b6b 100644
@@ -37,3 +36,3 @@
-+ * the load-acquire at A1. This release ensures that all updates to *ht
-+ * and the ring array made by this thread become visible to the opposing
-+ * thread once the tail value written here is observed.
++ * the load-acquire at A1/A3. This release ensures that all updates to
++ * *ht and the ring array made by this thread become visible to the
++ * opposing thread once the tail value written here is observed.
@@ -62 +61 @@
-+ rte_memory_order memorder)
++ int memorder)
@@ -78 +77 @@
-@@ -80,11 +98,9 @@ __rte_ring_hts_move_head(struct rte_ring_hts_headtail *d,
+@@ -58,13 +76,11 @@ __rte_ring_hts_move_prod_head(struct rte_ring *r, unsigned int num,
@@ -80 +79 @@
- uint32_t *entries)
+ uint32_t *free_entries)
@@ -83 +82 @@
-+ uint32_t n, stail;
++ uint32_t n, cons_tail;
@@ -86 +85,3 @@
-- op.raw = rte_atomic_load_explicit(&d->ht.raw, rte_memory_order_acquire);
+ const uint32_t capacity = r->capacity;
+
+- op.raw = rte_atomic_load_explicit(&r->hts_prod.ht.raw, rte_memory_order_acquire);
@@ -91 +92 @@
-@@ -94,7 +110,20 @@ __rte_ring_hts_move_head(struct rte_ring_hts_headtail *d,
+@@ -74,7 +90,20 @@ __rte_ring_hts_move_prod_head(struct rte_ring *r, unsigned int num,
@@ -95 +96 @@
-- __rte_ring_hts_head_wait(d, &op);
+- __rte_ring_hts_head_wait(&r->hts_prod, &op);
@@ -102 +103 @@
-+ op = __rte_ring_hts_head_wait(d, rte_memory_order_acquire);
++ op = __rte_ring_hts_head_wait(&r->hts_prod, rte_memory_order_acquire);
@@ -109 +110 @@
-+ stail = rte_atomic_load_explicit(&s->tail, rte_memory_order_acquire);
++ cons_tail = rte_atomic_load_explicit(&r->cons.tail, rte_memory_order_acquire);
@@ -113,2 +114,2 @@
-@@ -102,7 +131,7 @@ __rte_ring_hts_move_head(struct rte_ring_hts_headtail *d,
- * *old_head > cons_tail). So 'entries' is always between 0
+@@ -82,7 +111,7 @@ __rte_ring_hts_move_prod_head(struct rte_ring *r, unsigned int num,
+ * *old_head > cons_tail). So 'free_entries' is always between 0
@@ -117,2 +118,2 @@
-- *entries = capacity + s->tail - op.pos.head;
-+ *entries = capacity + stail - op.pos.head;
+- *free_entries = capacity + r->cons.tail - op.pos.head;
++ *free_entries = capacity + cons_tail - op.pos.head;
@@ -121,2 +122,2 @@
- if (unlikely(n > *entries))
-@@ -116,14 +145,17 @@ __rte_ring_hts_move_head(struct rte_ring_hts_headtail *d,
+ if (unlikely(n > *free_entries))
+@@ -96,13 +125,16 @@ __rte_ring_hts_move_prod_head(struct rte_ring *r, unsigned int num,
@@ -136 +137,65 @@
- } while (rte_atomic_compare_exchange_strong_explicit(&d->ht.raw,
+ } while (rte_atomic_compare_exchange_strong_explicit(&r->hts_prod.ht.raw,
+ (uint64_t *)(uintptr_t)&op.raw, np.raw,
+- rte_memory_order_acquire, rte_memory_order_acquire) == 0);
++ rte_memory_order_release, rte_memory_order_relaxed) == 0);
+
+ *old_head = op.pos.head;
+ return n;
+@@ -116,11 +148,9 @@ __rte_ring_hts_move_cons_head(struct rte_ring *r, unsigned int num,
+ enum rte_ring_queue_behavior behavior, uint32_t *old_head,
+ uint32_t *entries)
+ {
+- uint32_t n;
++ uint32_t n, prod_tail;
+ union __rte_ring_hts_pos np, op;
+
+- op.raw = rte_atomic_load_explicit(&r->hts_cons.ht.raw, rte_memory_order_acquire);
+-
+ /* move cons.head atomically */
+ do {
+ /* Restore n as it may change every loop */
+@@ -131,14 +161,27 @@ __rte_ring_hts_move_cons_head(struct rte_ring *r, unsigned int num,
+ * make sure that we read cons head/tail *before*
+ * reading prod tail.
+ */
+- __rte_ring_hts_head_wait(&r->hts_cons, &op);
++ /*
++ * A2: Synchronizes with the CAS at R2.
++ * Establishes a happens-before relationship with a thread of the same
++ * type that released the ht.raw, ensuring this thread observes all of
++ * its memory effects needed to maintain a safe partial order.
++ */
++ op = __rte_ring_hts_head_wait(&r->hts_cons, rte_memory_order_acquire);
++
++ /*
++ * A3: Establish a synchronizes-with edge using a store-release at R0.
++ * This ensures that all memory effects from the preceding opposing
++ * thread are observed.
++ */
++ prod_tail = rte_atomic_load_explicit(&r->prod.tail, rte_memory_order_acquire);
+
+ /* The subtraction is done between two unsigned 32bits value
+ * (the result is always modulo 32 bits even if we have
+ * cons_head > prod_tail). So 'entries' is always between 0
+ * and size(ring)-1.
+ */
+- *entries = r->prod.tail - op.pos.head;
++ *entries = prod_tail - op.pos.head;
+
+ /* Set the actual entries for dequeue */
+ if (n > *entries)
+@@ -151,13 +194,16 @@ __rte_ring_hts_move_cons_head(struct rte_ring *r, unsigned int num,
+ np.pos.head = op.pos.head + n;
+
+ /*
+- * this CAS(ACQUIRE, ACQUIRE) serves as a hoist barrier to prevent:
+- * - OOO reads of prod tail value
+- * - OOO copy of elems from the ring
++ * R2: Establishes a synchronizes-with edge with the load-acquire
++ * of ht.raw at A2. This makes sure that the store-release to the
++ * tail by this thread, if it was of the opposite type, becomes
++ * visible to another thread of the current type. That thread will
++ * then observe the updates in the same order, keeping a safe
++ * partial order.
+ */
+ } while (rte_atomic_compare_exchange_strong_explicit(&r->hts_cons.ht.raw,
@@ -138,4 +203,2 @@
-- rte_memory_order_acquire,
-- rte_memory_order_acquire) == 0);
-+ rte_memory_order_release,
-+ rte_memory_order_relaxed) == 0);
+- rte_memory_order_acquire, rte_memory_order_acquire) == 0);
++ rte_memory_order_release, rte_memory_order_relaxed) == 0);
More information about the stable
mailing list