[dpdk-dev] dmadev discussion summary

[Jerin Jacob]

On Sat, Jul 3, 2021 at 2:23 PM Morten Brørup <mb at smartsharesystems.com> wrote:
>
> > From: fengchengwen [mailto:fengchengwen at huawei.com]
> > Sent: Saturday, 3 July 2021 02.32
> >
> > On 2021/7/2 22:57, Morten Brørup wrote:
> > >> In the DPDK framework, many data-plane API names contain queues.
> > e.g.
> > >> eventdev/crypto..
> > >> The concept of virt queues has continuity.
> > >
> > > I was also wondering about the name "virtual queue".
> > >
> > > Usually, something "virtual" would be an abstraction of something
> > physical, e.g. a software layer on top of something physical.
> > >
> > > Back in the days, a "DMA channel" used to mean a DMA engine on a CPU.
> > If a CPU had 2 DMA channels, they could both be set up simultaneously.
> > >
> > > The current design has the "dmadev" representing a CPU or other chip,
> > which has one or more "HW-queues" representing DMA channels (of the
> > same type), and then "virt-queue" as a software abstraction on top, for
> > using a DMA channel in different ways through individually configured
> > contexts (virt-queues).
> > >
> > > It makes sense to me, although I would consider renaming "HW-queue"
> > to "channel" and perhaps "virt-queue" to "queue".
> >
> > The 'DMA channel' is more used than 'DMA queue', at least google show
> > that there are at least 20+ times more.
> >
> > It's a good idea build the abstraction layer: queue <> channel <> dma-
> > controller.
> > In this way, the meaning of each layer is relatively easy to
> > distinguish literally.
> >
> > will fix in V2
> >
>
> After re-reading all the mails in this thread, I have found one more important high level detail still not decided:
>
> Bruce had suggested flattening the DMA channels, so each dmadev represents a DMA channel. And DMA controllers with multiple DMA channels will have to instantiate multiple dmadevs, one for each DMA channel.
>
> Just like a four port NIC instantiates four ethdevs.
>
> Then, like ethdevs, there would only be two abstraction layers: dmadev <> queue, where a dmadev is a DMA channel on a DMA controller.
>
> However, this assumes that the fast path functions on the individual DMA channels of a DMA controller can be accessed completely independently and simultaneously by multiple threads. (Otherwise, the driver would need to implement critical regions or locking around accessing the common registers in the DMA controller shared by the DMA channels.)
>
> Unless any of the DMA controller vendors claim that this assumption about independence of the DMA channels is wrong, I strongly support Bruce's flattening suggestion.

It is wrong from alteast octeontx2_dma PoV.

# The PCI device is DMA controller where the driver/device is
mapped.(As device driver is based on PCI bus, We dont want to have
vdev for this)
# The PCI device has HW queue(s)
# Each HW queue has different channels.

In the current configuration, we have only one queue per device and it
has 4 channels. 4 channels are not threaded safe as it is based on
single queue.

I think, if we need to flatten it, I think, it makes sense to have
dmadev <> channel (and each channel can have thread-safe capability
based on how it mapped on HW queues based on the device driver
capability).






>
> -Morten
>