[dpdk-dev] [PATCH v10 00/13] Introducing EAL Bus-Device-Driver Model

Shreyansh Jain shreyansh.jain at nxp.com
Wed Jan 18 15:05:16 CET 2017


Link to v9: [19]

:: Introduction ::

DPDK has been inherently a PCI inclined framework. Because of this, the
design of device tree (or list) within DPDK is also PCI inclined. A
non-PCI device doesn't have a way of being expressed without using hooks
started from EAL to PMD.

(Check 'Version Changes' section for changes)

:: Overview of the Proposed Changes ::

Assuming the below graph for a computing node:

        device A1
          |
  +==.===='==============.============+ Bus A.
     |                    `--> driver A11     \
  device A2                `-> driver A12      \______
                                                |CPU |
                                                /`````
        device B1                              /
          |                                   /
  +==.===='==============.============+ Bus B`
     |                    `--> driver B11
  device B2                `-> driver B12


 - One or more buses are connected to a CPU (or core)
 - One or more devices are conneted to a Bus
 - Drivers are running instances which manage one or more devices
 - Bus is responsible for identifying devices (and interrupt propogation)
 - Driver is responsible for initializing the device

In [15], model assumes that rte_bus would be the base class using which
all the bus implementations would instantiate the objects. This patches
takes a much more basic approach, on the same lines of rte_device/
rte_driver and rte_pci_device/rte_pci_driver.
This is based on various review comments as well as offline (IRC)
discussions.

 - rte_bus is an abstract class which includes basic methods supported by
   all buses.
 - specific implementation, for example for PCI rte_pci_bus, would extend
   this class to form their own buses, with their own bus specific device
   and driver list.
 - 

 +-----------------+
 |rte_pci_bus      |
 |+---------------+|
 ||rte_bus        ||
 |+---------------+|
 +-----------------+

And example implementation would look like:

  .--------------->+-------------------+
  |                |rte_pci_bus        |
  |                | +----------------+|
  |                | |rte_bus         <------.
  |                | | name           ||     |
  |                | | scan           ||     |
  |                | | probe          ||     |
  |                | | attach         ||     |
  |                | | detach         ||     |
  |                | +----------------+|     |
  |                | pci_driver_list   |     |
  |  .-------------->pci_device_list   |     |
  |  |             | ...               |     |
  |  |             +-------------------+     |
  |  |                                       |
  |  +-------------------+                   |
  |  |rte_pci_device     |                   |
  '----bus               |                   |
     | +----------------+|                   |
     | |rte_device      ||                   |
     | | bus --------------------------------'
     | | ...            ||
     | +----------------+|
     | ...               |
     +-------------------+


:: Brief about Patch Layout ::

0001~0002: Introducing the basic Bus model and associated test case
0003     : Split the PCI match into a separate function
0004     : Remove an unnecessary loop over drivers in device detach
0005~0006: Introduce bus->scan and bus->probe APIs
0007     : Integrate the bus with EAL, without removing existing device
           driver init/scan model.
0008     : Add PCI Bus over Bus patches introduced earlier
0009~0010: Update the Bus and PCI test cases for scanning/probing
0011     : Enable PCI bus and remove code for direct PCI scan/probe from
           EAL
0012     : Add device hotplugging over the bus and introduce PCI helpers
0013     : Remove the deprecation notice for rte_bus


:: Pending Changes/Caveats ::

1. This patchset only moves the PCI into a bus. And, that movement is also
   currently part of the EAL (lib/librte_eal/linux)
   Eventual aim is the PCI bus reside in driver/bus/pci

2. Though the implementation for bus is common for Linux and BSD, the PCI
   bus implementation has been done/tested only for Linux.

3. RTE_REGISTER_BUS has been declared with contructor priority of 101
    It is important that Bus is registered *before* drivers are registered.
    Only way I could find to assure that was via 
    __attribute(contructor(priority)) of GCC. I am not sure how it would
    behave on other compilers. Any suggestions?
    - One suggestion from David Marchand was to use global bus object
      handles, which I have not implemented for now. If that is common
      choice, I will change in v3.
4. Hotplugging has been introduced over Bus with a caveat - it would only
   work for ports/NICs already scanned and available in the bus list.

:: ToDo list ::

 - Bump to librte_eal version
 - Documentation continues to have references to some _old_ PCI symbols

:: References ::

[1] http://dpdk.org/ml/archives/dev/2016-November/050186.html
[2] http://dpdk.org/ml/archives/dev/2016-November/050622.html
[3] http://dpdk.org/ml/archives/dev/2016-November/050416.html
[4] http://dpdk.org/ml/archives/dev/2016-November/050567.html
[5] http://dpdk.org/ml/archives/dev/2016-November/050628.html
[6] http://dpdk.org/ml/archives/dev/2016-November/050415.html
[7] http://dpdk.org/ml/archives/dev/2016-November/050443.html
[8] http://dpdk.org/ml/archives/dev/2016-November/050624.html
[9] http://dpdk.org/ml/archives/dev/2016-November/050296.html
[10] http://dpdk.org/ml/archives/dev/2016-December/051349.html - v1
[12] http://dpdk.org/ml/archives/dev/2016-December/052092.html - v2
[13] http://dpdk.org/ml/archives/dev/2016-December/052381.html - v3
[14] http://dpdk.org/ml/archives/dev/2016-December/053302.html - v4
[15] http://dpdk.org/ml/archives/dev/2016-December/053315.html - v5
[16] http://dpdk.org/ml/archives/dev/2017-January/055120.html  - v6
[17] http://dpdk.org/ml/archives/dev/2017-January/055320.html  - v7
[18] http://dpdk.org/ml/archives/dev/2017-January/055398.html  - v8
[19] http://dpdk.org/ml/archives/dev/2017-January/055621.html  - v9

:: Version Changes ::
v10:
 - bump to librte_eal LIBABIVER
 - remove an unnecessary call to rte_pci_match
 - INFO->DEBUG movement of a log message
 - Checkpatch issue fix
 - deprecation notice removal for rte_bus

v9:
 - Removed comments over function definitions
 - Documentation fixes as per review comments
 - Split the test_pci and test_bus patches
 - Split the patches for generic bus changes and PCI level bus changes
 - Add bus_autotest as entry into autotest_data.py

v8:
 - fix return value bug in rte_eal_pci_attach and rte_eal_pci_detach

v7:
 - update to rte_pci_match for const parameters
 - remove unnecessary log messages in probe; moved _after_ matching
   of device and driver
 - bug fixes in attach/detach methods
 - PCI disable for debugging was missed (from rte_eal_pci_init) in v6

v6:
 - Rearchitecture to bring bus object parallel to rte_device/driver
   This majorly includes:
   -- rte_pci_bus class and pci_bus as its object for PCI
   -- bus->attach/detach (hotplugging)
   -- removing bus->match as that is local to an implementation
 - rename symbols rte_eal_bus_* to rte_bus_*
 - restructuring patches (order) for simplicity
 - update to test_pci

v5:
 - Fix checkpatch error in Patch 0003

v4:
 - rebase over master (eac901ce)
 - Fix a bug in test_bus while setup and cleanup
 - rename rte_eal_get_bus to rte_eal_bus_get
 - Add helper (iterator) macros for easy bus,device,driver traversal
 - removed 0001 patch as it is already merged in master
 - fix missing rte_eal_bus_insert_device symbol in map file

v3:
 - rebase over master (c431384c8f)
 - revert patch 0001 changes for checkpatch (container_of macro)
 - qat/rte_qat_cryptodev update for rte_driver->probe
 - test_pci update for using a test_pci_bus for verification
 - some bug fixes based on internal testing.
 -- rte_eal_dev_attach not handling devargs
 -- blacklisting not working

v2:
 - No more bus->probe()
   Now, rte_eal_bus_probe() calls rte_driver->probe based on match output
 - new functions, rte_eal_pci_probe and rte_eal_pci_remove have been
   added as glue code between PCI PMDs and PCI Bus
   `-> PMDs are updated to use these new functions as callbacks for
       rte_driver
 - 'default' keyword has been removed from match and scan
 - Fix for incorrect changes in mlx* and nicvf*
 - Checkpatch fixes
 - Some variable checks have been removed from internal functions;
   functions which are externally visible continue to have such checks
 - Some rearrangement of patches:
   -- changes to drivers have been separated from EAL changes (but this
      does make PCI PMDs non-working for a particular patch)

Shreyansh Jain (13):
  eal/bus: introduce bus abstraction
  test: add basic bus infrastructure tests
  pci: split match and probe function
  eal: remove loop over drivers in device detach
  eal/bus: support for scanning of bus
  eal/bus: introduce support for bus probing
  eal: integrate bus scan and probe with EAL
  eal/pci: add support for PCI bus
  test: add test cases for scan and probe on BUS
  test: add Bus based scan and probe test cases for PCI
  eal: enable PCI bus
  eal: enable hotplugging of devices on bus
  doc: remove deprecation notice for rte_bus

 app/test/Makefile                               |   2 +-
 app/test/autotest_data.py                       |   6 +
 app/test/test.h                                 |   2 +
 app/test/test_bus.c                             | 686 ++++++++++++++++++++++++
 app/test/test_pci.c                             | 164 ++++--
 doc/guides/rel_notes/deprecation.rst            |   5 -
 lib/librte_eal/bsdapp/eal/Makefile              |   3 +-
 lib/librte_eal/bsdapp/eal/eal.c                 |  13 +-
 lib/librte_eal/bsdapp/eal/eal_pci.c             |  17 +
 lib/librte_eal/bsdapp/eal/rte_eal_version.map   |  15 +-
 lib/librte_eal/common/Makefile                  |   2 +-
 lib/librte_eal/common/eal_common_bus.c          | 138 +++++
 lib/librte_eal/common/eal_common_dev.c          |  56 +-
 lib/librte_eal/common/eal_common_pci.c          | 368 ++++++++-----
 lib/librte_eal/common/eal_private.h             |  10 -
 lib/librte_eal/common/include/rte_bus.h         | 191 +++++++
 lib/librte_eal/common/include/rte_dev.h         |   1 +
 lib/librte_eal/common/include/rte_pci.h         | 148 ++++-
 lib/librte_eal/linuxapp/eal/Makefile            |   3 +-
 lib/librte_eal/linuxapp/eal/eal.c               |  13 +-
 lib/librte_eal/linuxapp/eal/eal_pci.c           |  57 +-
 lib/librte_eal/linuxapp/eal/rte_eal_version.map |  15 +-
 22 files changed, 1631 insertions(+), 284 deletions(-)
 create mode 100644 app/test/test_bus.c
 create mode 100644 lib/librte_eal/common/eal_common_bus.c
 create mode 100644 lib/librte_eal/common/include/rte_bus.h

-- 
2.7.4



More information about the dev mailing list