[dpdk-dev] [RFC ][PATCH] Introduce RTE_ARCH_STRONGLY_ORDERED_MEM_OPS configuration parameter

[Richardson, Bruce]

> -----Original Message-----
> From: dev [mailto:dev-bounces at dpdk.org] On Behalf Of Jerin Jacob
> Sent: Tuesday, November 3, 2015 4:53 PM
> To: Ananyev, Konstantin <konstantin.ananyev at intel.com>
> Cc: dev at dpdk.org
> Subject: Re: [dpdk-dev] [RFC ][PATCH] Introduce
> RTE_ARCH_STRONGLY_ORDERED_MEM_OPS configuration parameter
> 
> On Tue, Nov 03, 2015 at 04:28:00PM +0000, Ananyev, Konstantin wrote:
> >
> >
> > > -----Original Message-----
> > > From: Jerin Jacob [mailto:jerin.jacob at caviumnetworks.com]
> > > Sent: Tuesday, November 03, 2015 4:19 PM
> > > To: Ananyev, Konstantin
> > > Cc: dev at dpdk.org
> > > Subject: Re: [dpdk-dev] [RFC ][PATCH] Introduce
> > > RTE_ARCH_STRONGLY_ORDERED_MEM_OPS configuration parameter
> > >
> > > On Tue, Nov 03, 2015 at 03:57:24PM +0000, Ananyev, Konstantin wrote:
> > > >
> > > >
> > > > > -----Original Message-----
> > > > > From: dev [mailto:dev-bounces at dpdk.org] On Behalf Of Jerin Jacob
> > > > > Sent: Tuesday, November 03, 2015 3:52 PM
> > > > > To: dev at dpdk.org
> > > > > Subject: [dpdk-dev] [RFC ][PATCH] Introduce
> > > > > RTE_ARCH_STRONGLY_ORDERED_MEM_OPS configuration parameter
> > > > >
> > > > > rte_ring implementation needs explicit memory barrier in weakly
> > > > > ordered architecture like ARM unlike strongly ordered
> > > > > architecture like X86
> > > > >
> > > > > Introducing RTE_ARCH_STRONGLY_ORDERED_MEM_OPS configuration to
> > > > > abstract such dependency so that other weakly ordered
> > > > > architectures can reuse this infrastructure.
> > > >
> > > > Looks a bit clumsy.
> > > > Please try to follow this suggestion instead:
> > > > http://dpdk.org/ml/archives/dev/2015-October/025505.html
> > >
> > > Make sense. Do we agree on a macro that is defined based upon
> > > RTE_ARCH_STRONGLY_ORDERED_MEM_OP to remove clumsy #ifdef every where ?
> >
> > Why do we need that macro at all?
> > Why just not have architecture specific macro as was discussed in that
> thread?
> >
> > So for intel somewhere inside
> > lib/librte_eal/common/include/arch/x86/rte_atomic.h
> >
> > it would be:
> >
> > #define rte_smp_wmb()	rte_compiler_barrier()
> >
> > For arm inside lib/librte_eal/common/include/arch/x86/rte_atomic.h
> >
> > #define rte_smp_wmb()	rte_wmb()
> 
> I am not sure about the other architecture but in armv8 device memory
> (typically mapped through NIC PCIe BAR space) are strongly ordered.
> So there is one more dimension to the equation(normal memory or device
> memory).
> IMO rte_smp_wmb() -> rte_wmb() mapping to deal with device memory may not
> be correct on arm64 ?
> 
> Thoughts ?
> 
In cases like that I don't think barriers are needed on any platform so the proposed scheme will work fine. It's up the driver writer to know about when they are writing to device BARs or not.

/Bruce