[PATCH v4] usertools: rewrite pmdinfo

[Ferruh Yigit]

On 9/20/2022 11:42 AM, Robin Jarry wrote:

> 
> dpdk-pmdinfo.py does not produce any parseable output. The -r/--raw flag
> merely prints multiple independent JSON lines which cannot be fed
> directly to any JSON parser. Moreover, the script complexity is rather
> high for such a simple task: extracting PMD_INFO_STRING from .rodata ELF
> sections. Rewrite it so that it can produce valid JSON.
> 
> Remove the PCI database parsing for PCI-ID to Vendor-Device names
> conversion. This should be done by external scripts (if really needed).
> 
> Here are some examples of use with jq:
> 
> Get the complete info for a given driver:
> 
>   ~$ usertools/dpdk-pmdinfo.py build/app/dpdk-testpmd | \
>     jq '.[] | select(.name == "dmadev_idxd_pci")'
>   {
>     "name": "dmadev_idxd_pci",
>     "params": "max_queues=0",
>     "kmod": "vfio-pci",
>     "pci_ids": [
>       {
>         "vendor": "8086",
>         "device": "0b25"
>       }
>     ]
>   }
> 
> Get only the required kernel modules for a given driver:
> 
>   ~$ usertools/dpdk-pmdinfo.py build/app/dpdk-testpmd | \
>     jq '.[] | select(.name == "net_i40e").kmod'
>   "* igb_uio | uio_pci_generic | vfio-pci"
> 
> Get only the required kernel modules for a given device:
> 
>   ~$ usertools/dpdk-pmdinfo.py build/app/dpdk-testpmd | \
>     jq '.[] | select(.pci_ids[] | .vendor == "15b3" and .device == "1013").kmod'
>   "* ib_uverbs & mlx5_core & mlx5_ib"
> 
> Print the list of drivers which define multiple parameters without
> space separators:
> 
>   ~$ usertools/dpdk-pmdinfo.py build/app/dpdk-testpmd | \
>     jq '.[] | select(.params!=null and (.params|test("=[^ ]+="))) | {name, params}'
>   ...
> 
> The script passes flake8, black, isort and pylint checks.
> 
> I have tested this with a matrix of python/pyelftools versions:
> 
>                               pyelftools
>                 0.22 0.23 0.24 0.25 0.26 0.27 0.28 0.29
>           3.6     ok   ok   ok   ok   ok   ok   ok   ok
>           3.7     ok   ok   ok   ok   ok   ok   ok   ok
>    Python 3.8     ok   ok   ok   ok   ok   ok   ok   ok
>           3.9     ok   ok   ok   ok   ok   ok   ok   ok
>           3.10  fail fail fail fail   ok   ok   ok   ok
> 
> All failures with python 3.10 are related to the same issue:
> 
>    File "elftools/construct/lib/container.py", line 5, in <module>
>      from collections import MutableMapping
>    ImportError: cannot import name 'MutableMapping' from 'collections'
> 
> Python 3.10 support is only available since pyelftools 0.26. The script
> will only work with Python 3.6 and later. Update the minimal system
> requirements and release notes.
> 
> NB: The output produced by the legacy -r/--raw flag can be obtained with
> the following command:
> 
>    strings build/app/dpdk-testpmd | sed -n 's/^PMD_INFO_STRING= //p'
> 
> Cc: Olivier Matz <olivier.matz at 6wind.com>
> Cc: Ferruh Yigit <ferruh.yigit at xilinx.com>
> Cc: Bruce Richardson <bruce.richardson at intel.com>
> Signed-off-by: Robin Jarry <rjarry at redhat.com>

<...>

> diff --git a/doc/guides/rel_notes/release_22_11.rst b/doc/guides/rel_notes/release_22_11.rst
> index 8c021cf0505e..67054f5acdc9 100644
> --- a/doc/guides/rel_notes/release_22_11.rst
> +++ b/doc/guides/rel_notes/release_22_11.rst
> @@ -84,6 +84,11 @@ API Changes
>      Also, make sure to start the actual text at the margin.
>      =======================================================
> 
> +* The ``dpdk-pmdinfo.py`` script was rewritten to produce valid JSON only.
> +  PCI-IDs parsing has been removed.
> +  To get a similar output to the (now removed) ``-r/--raw`` flag, you may use the following command::
> +
> +     strings $dpdk_binary_or_driver | sed -n 's/^PMD_INFO_STRING= //p'
> 

Empty line is missing (in case there will be a new version for some 
other reason).


Thanks for the update,
Tested-by: Ferruh Yigit <ferruh.yigit at xilinx.com>