[dpdk-dev] [dpdk-techboard] RFC: DPDK drivers for DPDK bus types

Bruce Richardson bruce.richardson at intel.com
Thu Mar 11 17:45:30 CET 2021


On Thu, Mar 11, 2021 at 04:40:14PM +0000, Bruce Richardson wrote:
> On Thu, Mar 11, 2021 at 04:44:49PM +0100, Thomas Monjalon wrote:
> > 11/03/2021 16:19, Bruce Richardson:
> > > Hi all,
> > > 
> > > looking for input here into the area of bus-type drivers and interaction
> > > with other drivers in DPDK.
> > > 
> > > By way of context, I'm looking at extending the vdev support in the
> > > "raw/ioat" driver (file raw/ioat/idxd_vdev.c) to make it more user
> > > friendly.  These devices are accessed by DPDK via nodes in /dev and
> > > paths in /sys, with the vdev parameters being passed identifying the
> > > particular devices to use. However, the presence of these devices can be
> > > detected at runtime by a scan of /dev and /sys, and so it's easy enough
> > > to implement a custom bus-type driver in DPDK to detect these, rather
> > > than having the user pass in vdev parameters (which can get awkward to
> > > use as the number of devices increases).
> > 
> > I agree. vdev bus should be used for creating device from the void.
> > If the device has its roots in the system (HW or SW), there should be
> > a bus for that.
> > 
> > 
> > > However, looking through a few other drivers in the "bus" directory, it
> > > appears that scanning system paths, i.e. /sys, is fairly common, so I'm
> > > wondering if it's possible to have some sharing of functionality here.
> > > Unfortunately, the use of /sys in each of these drivers I've looked at
> > > seem sufficiently different to me that I've not immediately seen a
> > > common level of abstraction we can use. Therefore I'm looking for
> > > suggestions here that those in the community might have.
> > 
> > Not sure it's worth looking for such sharing between bus.
> > 
> > 
> > > On a related note, I'm also concerned about the need for a single device
> > > type, e.g. one used by DPDK and shared with the kernel, to require two
> > > separate drivers to work together to support it - a bus driver for
> > > scanning and a type-specific driver for the actual functional
> > > implementation. Can we not find a way to reduce the number of drivers
> > > needing to be supported?
> > 
> > The bus driver is managing the device life.
> > The device driver implements a functional class.
> > I don't see what to save.
> > Maybe you are biased because the rawdev class is fake.
> > 
> 
> The fact of the rawdev class being fake is irrelevant to this, the same
> would apply to any device class which as you say - "has it's roots in the
> system". However, for some of these devices, the roots in the system may
> well be unique, and therefore - irrespective of device class - one needs
> two separate drivers to work with such a device, a bus driver to do system
> scanning, and a device driver to manage the instances.
> 
> > 
> > > Following on from this, and if we can't find a good way of doing a
> > > generic driver for scanning /sys nodes, I wonder if there is value in
> > > providing a "generic" bus implementation in DPDK, as a catch-all for
> > > device drivers which need their own custom probing, but that do not
> > > neatly fall into the other types. The way this might work is to have the
> > > scanning and probing of devices left entirely to the device driver
> > > implementation itself. For example, rather than creating an idxd
> > > bus driver, it would be easier and more self-contained to have the
> > > rawdev driver itself able to perform scanning and probing - keeping the
> > > code all in one place. All the bus driver would have to do is maintain a
> > > list of drivers and found devices reported by the individual driver
> > > after they have done their own probing.
> > 
> > So you mean there is a single user of the bus,
> > so the implementation could be moved into the device driver,
> > relying on a fake bus?
> > 
> 
> Exactly. If the bus exists for only one device instance - and will only
> ever exist for one device - why not allow the driver itself to have an
> extra function called "scan", rather than having to create a custom bus
> driver just to do that scanning.
> 
> > 
> > > Other possible candidates to think about that might be able to use
> > > their own probing from a generic bus might be, e.g. af_xdp driver, or a
> > > TAP or memif driver.
> > 
> > These devices don't exist naturally in the system, so I think they should
> > be vdev.
> > 
> AF_XDP devices certainly exist in the system, and serves at least as a good
> theoretical model of exactly what I am describing. Drivers for TAP or memif
> could be written to scan for and attach to already-existing instances on
> the system too. 
> 
> For the AF_XDP case, suppose that the kernel was extended to allow for
> certain queues to be marked or named, one could then create an AF_XDP bus
> driver to look for named queues on each NIC interface. Similarly a TAP bus
> device could scan for TAP instances previously created and configured and
> attach to them. The key thing is that in each case, the scanning done is
> unique and applies to only one net driver - nothing rawdev related about
> this! Therefore, while I don't think we would ever really implement bus
> drivers for these specific cases, they are examples which I think shows the
> value in having a bus type to allow generic scanning, rather than forcing a
> new bus driver to be created for each individual type of scanning
> necessary.
> 

Just to finally add that I'm happy enough to do a bus driver for the
specific case I'm looking at - it's actually far less work, I believe.
However, I'm just trying to avoid us having more components to maintain in
the future.

/Bruce


More information about the dev mailing list