[dpdk-dev] [PATCH 6/6] doc: introduction to prgdev

Bruce Richardson bruce.richardson at intel.com
Tue Mar 7 12:12:31 CET 2017


On Tue, Mar 07, 2017 at 10:34:06AM +0000, Chen, Jing D wrote:
> Hi, Thomas,
> 
> > -----Original Message-----
> > From: Thomas Monjalon [mailto:thomas.monjalon at 6wind.com]
> > Sent: Monday, March 6, 2017 11:27 PM
> > To: Chen, Jing D <jing.d.chen at intel.com>
> > Cc: dev at dpdk.org; Liang, Cunming <cunming.liang at intel.com>; Rogers,
> > Gerald <gerald.rogers at intel.com>; Wiles, Keith <keith.wiles at intel.com>;
> > Richardson, Bruce <bruce.richardson at intel.com>; Mcnamara, John
> > <john.mcnamara at intel.com>
> > Subject: Re: [dpdk-dev] [PATCH 6/6] doc: introduction to prgdev
> > 
> > 2017-03-02 12:03, Chen Jing D:
> > > +Overview
> > > +========
> > 
> > I think the first review pass of this series must be focused on the overview
> > you give here (thanks for the detailed explanations).
> > 
> > I'll try to sum up while commenting.
> > 
> > [...]
> > The target is programmable devices.
> > 
> > > +The major purpose of prgdev is to help DPDK users to load/upgrade RTL
> > > +images for FPGA devices, or upgrade firmware for programmble NICs.
> > 
> > This is a control plane feature.
> > You want to have it in DPDK to allow dynamic programmation while running
> > the device, right?
> 
> Exactly right.
> 
> > 
> > [...]
> > > +When the set of APIs is introduced, a general question is why we need
> > > +it in DPDK community?
> > 
> > Good question :)
> > 
> > [...]
> > > +Any devices, having the capability to store/load a piece of info
> > > +to/from the deivce then changed hardware behavior, and applicable to
> > > +prgdev programming model, could be registered as a prgdev device.
> > > +
> > > +The device can be programmed (dynamic) within DPDK or have been prior
> > > +programmed
> > > +(static) prior to a DPDK application being launched.
> > [...]
> > > +Besides the simple API to upload/download image, the prgdev also
> > > +introduces a mechanism internally to switch drivers and
> > > +register/unregister device on the fly. With this mechanism, apps can
> > > +use the programmable device, unbind other personalities on demand,
> > then program it and bind it back with new personalities.
> > > +Apps can follow below examples to simply complete the whole process.
> > 
> > I disagree about the specific bind/unbind for prgdev.
> > We must improve binding inside DPDK for every devices.
> 
> First of all, let us try to imagine what is the safe status for device before apps
> intending to download some image to it - apps wouldn't operate on the device, 
> any behaviors like configuring registers, receive/transmit data may impair the
> device or make the download failed. 
> Following first answer to prevent app accessing device during image
> downloading, how can we achieve that? Detach drivers with device is a smart
> idea, right? But the problem is how can apps use prgdev API to download image
> after all drivers detached with the device?
> So, the final question is how can we just detached others driver except prgdev
> one? I can't find answer in current DPDK framework, that's why I'd like to introduce
> bind/unbind functions to detach other drivers from the device.
> 
> I'm open to this problem. If any suggestion or mechanism can help on it, I can
> remove these 2.
> 
> BTW, not all devices or image download actions need the device to detach the 
> device, depending on hardware implementation.
> 
> > 
> > > +Note that bind/unbind actions are different concept from that a whole
> > > +device attach/detach. For example, ``rte_eal_dev_detach()``, which
> > > +will try to detach the drivers with device and remove the whole
> > > +device from specific class of devices (ethdev, for example). Then, the
> > whole device is invisible until a new 'probe'
> > > +is activated.
> > 
> > I do not understand.
> 
> See above explanations.
> 
> > 
> > > +During the whole procedure of image upload/download, prgdev handler
> > > +is always valid and apps can use it operate on programmable device.
> > > +The reason why unbind is necessary is it may break the hardware when
> > > +programming is in progress while other functions are active. Using
> > > +programmble NIC as an example, it's a little impossible to
> > > +receive/forward packets for hardware while updating firmware. In this
> > > +case, apps need to detach ethdev function before firmware upgrading.
> > > +Again, prgdev handler is still valid, it's function-level detach,
> > > +different from device-level detach. After finishing image download,
> > > +original function needs to attach back, either use same or different
> > > +drivers, depends on personalities changed or not. For a programmble NIC,
> > the driver may be the same. For FPGA, it may not, see below examples to get
> > more details.
> > 
> > If the personality of the device is changed, it must be seen as a new device
> > from e.g. the ethdev point of view, while keeping the same prgdev device.
> > In other words, a device can have several interfaces at the same time (ethdev,
> > cryptodev, eventdev, prgdev, whatever).
> > I think we can dynamically create/destroy some interfaces while keeping
> > track of the underlying device.
> 
> Fully agree. But current PCI device with 'vendor_id' and 'device_id' ony can
> be bind to single driver at a time. So I add 'rte_prgdev_allocate/release' to
> support register/unregister prgdev dynamically without BDF.
> 
> > 
> > > +Another reason to provide bind/unbind action is programmble devices,
> > > +like FPGA, are not identified driver by 'vendor ID' and 'device ID',
> > > +they might not be changed in all the ways, even FPGA is fully
> > > +programmed. So, it depends on internal mechanism of FPGA, not 'vendor
> > > +ID' and 'device ID' to identify proper drivers, either a register
> > > +value or signature, depending on FPGA hardware design. In this case,
> > > +EAL or other bus driver doesn't have the capability to identify
> > > +proper driver for FPGA device. With prgdev introduced, while FPGA is
> > > +always a prgdev, FPGA can use prgdev as primary driver, to find proper
> > function driver.
> > 
> > You mean prgdev should help the bus layer to map the right driver interface?
> > It looks weird and dangerous. The standard way to identify a PCI device is to
> > look at its IDs. Other unknown methods must be strongly discussed.
> 
> For programmable Ethernet device, it's not truce. But for FPGA, it's. When FPGA
> is produced, the device ID indicate what model it is and won't be changed
> anyway, even being reprogrammed. It used some not-generic mechanism, like
> AFU id to distinguish the personalities. So, for FPGA, a prgdev driver can be used
> as primary driver to identify personalities and then register to specific devices.

Sounds like we would need an FPGA bus driver in that case. I think that
would be a better solution than having a specific device driver loading
other drivers.

/Bruce


More information about the dev mailing list