[PATCH] usertools: rewrite pmdinfo

[Ferruh Yigit]

On 9/13/2022 11:58 AM, Robin Jarry wrote:
> dpdk-pmdinfo.py does not produce any parseable output. The -r/--raw flag
> merely prints multiple independent JSON lines which cannot be fed
> directly to any JSON parser. Moreover, the script complexity is rather
> high for such a simple task: extracting PMD_INFO_STRING from .rodata ELF
> sections. Rewrite it so that it can produce valid JSON.
> 
> Remove the PCI database parsing for PCI-ID to Vendor-Device names
> conversion. This should be done by external scripts (if really needed).
> 
> Here are some examples of use with jq:
> 
> Get the complete info for a given driver:
> 
>   ~$ usertools/dpdk-pmdinfo.py build/app/dpdk-testpmd | \
>     jq '.[] | select(.name == "dmadev_idxd_pci")'
>   {
>     "name": "dmadev_idxd_pci",
>     "params": "max_queues=0",
>     "kmod": "vfio-pci",
>     "devices": [
>       {
>         "vendor_id": "8086",
>         "device_id": "0b25",
>         "subsystem_device_id": "ffff",
>         "subsystem_system_id": "ffff"
>       }
>     ]
>   }
> 
> Get only the required kernel modules for a given driver:
> 
>   ~$ usertools/dpdk-pmdinfo.py build/app/dpdk-testpmd | \
>     jq '.[] | select(.name == "net_i40e").kmod'
>   "* igb_uio | uio_pci_generic | vfio-pci"
> 
> Get only the required kernel modules for a given device:
> 
>   ~$ usertools/dpdk-pmdinfo.py build/app/dpdk-testpmd | \
>     jq '.[] | select(.devices[] | .vendor_id == "15b3" and .device_id == "1013").kmod'
>   "* ib_uverbs & mlx5_core & mlx5_ib"
> 
> Print the list of drivers which define multiple parameters without
> string separators:
> 
>   ~$ usertools/dpdk-pmdinfo.py build/app/dpdk-testpmd | \
>     jq '.[] | select(.params!=null and (.params|test("=[^ ]+="))) | {name, params}'
>   ...
> 
> The script passes flake8, black, isort and pylint checks.
> 
> I have tested this with a matrix of python/pyelftools versions:
> 
>                               pyelftools
>                 0.22 0.23 0.24 0.25 0.26 0.27 0.28 0.29
>           3.6     ok   ok   ok   ok   ok   ok   ok   ok
>           3.7     ok   ok   ok   ok   ok   ok   ok   ok
>    Python 3.8     ok   ok   ok   ok   ok   ok   ok   ok
>           3.9     ok   ok   ok   ok   ok   ok   ok   ok
>           3.10  fail fail fail fail   ok   ok   ok   ok
> 
> All failures with python 3.10 are related to the same issue:
> 
>    File "elftools/construct/lib/container.py", line 5, in <module>
>      from collections import MutableMapping
>    ImportError: cannot import name 'MutableMapping' from 'collections'
> 
> Python 3.10 support is only available since pyelftools 0.26.
> 
> NB: The output produced by the legacy -r/--raw flag can be obtained with
> the following command:
> 
>    strings build/app/dpdk-testpmd | sed -n 's/^PMD_INFO_STRING= //p'
> 
> Cc: Olivier Matz<olivier.matz at 6wind.com>
> Signed-off-by: Robin Jarry<robin at jarry.cc>

Hi Robin,

Thanks for the work.

One of the major usecase of the script is to get information from binary 
drivers. So intentions of the script is to run it on drivers more than 
applications (dpdk-testpmd).

When I run it on one of the .so drivers, it is generating some warnings 
[1], is this expected?

[1]
$ ./usertools/dpdk-pmdinfo.py ./build/drivers/librte_net_ixgbe.so 
 

warning: cannot find librte_ethdev.so.23
warning: cannot find librte_eal.so.23
warning: cannot find librte_kvargs.so.23
warning: cannot find librte_telemetry.so.23
warning: cannot find librte_net.so.23
warning: cannot find librte_mbuf.so.23
warning: cannot find librte_mempool.so.23
warning: cannot find librte_ring.so.23
warning: cannot find librte_meter.so.23
warning: cannot find librte_bus_pci.so.23
warning: cannot find librte_pci.so.23
warning: cannot find librte_bus_vdev.so.23
warning: cannot find librte_hash.so.23
warning: cannot find librte_rcu.so.23
warning: cannot find librte_security.so.23
warning: cannot find librte_cryptodev.so.23
[
   {
     "name": "net_ixgbe_vf",
     "params": "pflink_fullchk=<0|1>",
     "kmod": "* igb_uio | vfio-pci",
     "devices": [
...
...