[dpdk-dev] [PATCHv2 4/4] pmdinfo.py: Add tool to query binaries for hw and other support information

[Neil Horman]

On Thu, May 19, 2016 at 12:02:27PM +0300, Panu Matilainen wrote:
> On 05/19/2016 12:08 AM, Neil Horman wrote:
> > This tool searches for the primer sting PMD_DRIVER_INFO= in any ELF binary,
> > and, if found parses the remainder of the string as a json encoded string,
> > outputting the results in either a human readable or raw, script parseable
> > format
> > 
> > Note that, in the case of dynamically linked applications, pmdinfo.py will scan
> > for implicitly linked PMDs by searching the specified binaries .dynamic section
> > for DT_NEEDED entries that contain the substring librte_pmd.  The DT_RUNPATH,
> > LD_LIBRARY_PATH, /usr/lib and /lib are searched for these libraries, in that
> > order
> 
> Scanning /usr/lib and /lib does little good on systems where /usr/lib64 and
> /lib64 are the standard path, such as x86_64 Fedora / RHEL and derivates.
> 
Ah, sorry, forgot the 64 bit variants, I can add those in.

> With the path changed (or LD_LIBRARY_PATH set manually), I can confirm it
> works for a shared binary which is --whole-archive linked to all of DPDK
> such as ovs-vswitchd currently is (because it needs to for static DPDK
> linkage and is not aware of plugin autoloading).
> 
Right, thats why it works, because DPDK always requires --whole-archive for
static linking, and likely always will (see commit
20afd76a504155e947c770783ef5023e87136ad8)

> It doesn't help testpmd though because its not linked with --whole-archive
> in the shared case, so its not working for the main DPDK executable...
> 
This sentence doesn't make sense --whole-archive is only applicable in the
static binary case, and only when linking archive files.

> In any case, using --whole-archive is a sledgehammer solution at best, and
> against the spirit of shared libs and plugins in particular.
> 
It may be a sledgehammer solution, but its the one dpdk uses, and will likely
use in perpituity.

> I think the shared linkage case can be solved by exporting the PMD path from
> librte_eal (either through an elf section or c-level symbol) and teach the
> script to detect the case of an executable dynamically linked to librte_eal,
> fish the path from there and then process everything in that path.
> 
I really disagree with this, because its a half-measure at best.  Yes, if its
set, you will definately get all the shared objects in that directory loaded,
but that is in no way a guarantee that those are the only libraries that get
loaded (the application may load its own independently).  So you're left in this
situation in which you get maybe some of the hardware support an application
offers.  Its also transient.  That is to say, if you configure a plugin
directory and search it when you scan an application, its contents may change
post scan, leading to erroneous results.

The way I see it, we have 3 cases that we need to handle:

1) Statically linked application - in this case, all pmds that are statically
linked in to the application will be reported, so we're good here

2) Dynamically loaded via DT_NEEDED entries - This is effectively the same as a
static linking case, in that we have a list of libraries that must be resolved
at run time, so we are safe to search for and scan the DSO's that the
application ennumerates

3) Dynamically loaded via dlopen - In this case, we don't actually know until
runtime what DSO's are going to get loaded, even if RTE_EAL_PMD_PATH is set,
because the contents of that path can change at arbitrary times.  In this case,
its correct to indicate that the application itself _doesn't_ actually support
the hardware of the PMD's in that path, because until the application is
executed, it has none of the support embodied in any DSO that it loads via
dlopen.  The hardware support travels with the DSO itself, and so its correct to
only display hardware support when the PMD shared library itself is scanned.

Handling case 3 the way I'm proposing is exactly the way the OS does it (that is
to say, it only details hardware support for the module being queried, and you
have to specify the module name to get that).  I don't see there being any
problem with that.

> > 
> > If a file is specified with no path, it is assumed to be a PMD DSO, and the
> > LD_LIBRARY_PATH, /usr/lib/ and /lib is searched for it
> 
> Same as above, /usr/lib/ and /lib is incorrect for a large number of
> systems.
> 
Yeah, I'll add 64 bit detection and correct that path accordingly

> > 
> > Currently the tool can output data in 3 formats:
> > 
> > a) raw, suitable for scripting, where the raw JSON strings are dumped out
> > b) table format (default) where hex pci ids are dumped in a table format
> > c) pretty, where a user supplied pci.ids file is used to print out vendor and
> > device strings
> 
> c) is a nice addition. Would be even nicer if it knew the most common
> pci.ids locations so it doesn't need extra arguments in those cases, ie see
> if /usr/share/hwdata/pci.ids or /usr/share/misc/pci.ids exists and use that
> unless overridden on the cli.
> 
Yeah, I can do that.

> 	- Panu -
> 
> 
>