rte_ring move head question for machines with relaxed MO (arm/ppc)

[Konstantin Ananyev]

> -----Original Message-----
> From: Konstantin Ananyev <konstantin.ananyev at huawei.com>
> Sent: Tuesday, October 8, 2024 4:46 PM
> To: Wathsala Wathawana Vithanage <wathsala.vithanage at arm.com>; dev at dpdk.org
> Cc: Honnappa Nagarahalli <Honnappa.Nagarahalli at arm.com>; jerinj at marvell.com; drc at linux.ibm.com; nd <nd at arm.com>
> Subject: RE: rte_ring move head question for machines with relaxed MO (arm/ppc)
> 
> 
> > > 1. rte_ring_generic_pvt.h:
> > > =====================
> > >
> > > pseudo-c-code                                      //        related armv8 instructions
> > > --------------------                                                 --------------------------------------
> > >  head.load()                                          //        ldr [head]
> > >  rte_smp_rmb()                                    //        dmb ishld
> > >  opposite_tail.load()                            //        ldr [opposite_tail]
> > >  ...
> > >  rte_atomic32_cmpset(head, ...)      //        ldrex[head];... stlex[head]
> > >
> > >
> > > 2. rte_ring_c11_pvt.h
> > > =====================
> > >
> > > pseudo-c-code                                       //        related armv8 instructions
> > > --------------------                                                 --------------------------------------
> > > head.atomic_load(relaxed)                 //        ldr[head]
> > > atomic_thread_fence(acquire)           //        dmb ish
> > > opposite_tail.atomic_load(acquire)   //        lda[opposite_tail]
> > > ...
> > > head.atomic_cas(..., relaxed)              //        ldrex[haed]; ... strex[head]
> > >
> > >
> > > 3.   rte_ring_hts_elem_pvt.h
> > > ==========================
> > >
> > > pseudo-c-code                                       //        related armv8 instructions
> > > --------------------                                                 --------------------------------------
> > > head.atomic_load(acquire)                //        lda [head]
> > > opposite_tail.load()                             //        ldr [opposite_tail]
> > > ...
> > > head.atomic_cas(..., acquire)            //         ldaex[head]; ... strex[head]
> > >
> > > The questions that arose from these observations:
> > > a) are all 3 approaches equivalent in terms of functionality?
> > Different, lda (Load with acquire semantics) and ldr (load) are different.
> 
> I understand that, my question was:
> lda {head]; ldr[tail]
> vs
> ldr [head]; dmb ishld; ldr [tail];
> 
> Is there any difference in terms of functionality (memory ops ordering/observability)?

To be more precise:

lda {head]; ldr[tail]
vs
ldr [head]; dmb ishld; ldr [tail]; 
vs
ldr [head]; dmb ishld; lda [tail];

what would be the difference between these 3 cases?

> 
> >
> > > b) if yes, is there any difference in terms of performance between:
> > >      "ldr; dmb; ldr;"   vs "lda; ldr;"
> > >       ?
> > dmb is a full barrier, performance is poor.
> > I would assume (haven't measured) ldr; dmb; ldr to be less performant than lda;ldr;
> 
> Through all this mail am talking about 'dmb ishld', sorry for not being clear upfront.
> 
> >
> > > c) Comapring at 1) and 2) above, combination of
> > >    ldr [head]; dmb; lda [opposite_tail]:
> > >    looks     like an overkill to me.  Wouldn't just:
> > >    ldr [head]; dmb; ldr[opposite_tail];
> > >    be sufficient here?
> > lda [opposite_tail]: synchronizes with stlr in tail update that happens after array update.
> > So, it cannot be changed to ldr.
> 
> Can you explain me a bit more here why it is not possible?
> From here:
> https://developer.arm.com/documentation/dui0802/b/A32-and-T32-Instructions/LDA-and-STL
> "There is no requirement that a load-acquire and store-release be paired."
> Do I misinterpret this statement somehow?
> 
> > lda can be replaced with ldapr (LDA with release consistency - processor consistency)
> > which is more performant as lda is allowed to rise above stlr. Can be done with -mcpu=+rcpc
> >
> > --wathsala
> >