[PATCH v3] usertools: rewrite pmdinfo

[Ferruh Yigit]

On 9/20/2022 10:08 AM, Robin Jarry wrote:
> CAUTION: This message has originated from an External Source. Please use proper judgment and caution when opening attachments, clicking links, or responding to this email.
> 
> 
> dpdk-pmdinfo.py does not produce any parseable output. The -r/--raw flag
> merely prints multiple independent JSON lines which cannot be fed
> directly to any JSON parser. Moreover, the script complexity is rather
> high for such a simple task: extracting PMD_INFO_STRING from .rodata ELF
> sections. Rewrite it so that it can produce valid JSON.
> 
> Remove the PCI database parsing for PCI-ID to Vendor-Device names
> conversion. This should be done by external scripts (if really needed).
> 
> Here are some examples of use with jq:
> 
> Get the complete info for a given driver:
> 
>   ~$ usertools/dpdk-pmdinfo.py build/app/dpdk-testpmd | \
>     jq '.[] | select(.name == "dmadev_idxd_pci")'
>   {
>     "name": "dmadev_idxd_pci",
>     "params": "max_queues=0",
>     "kmod": "vfio-pci",
>     "pci_ids": [
>       {
>         "vendor": "8086",
>         "device": "0b25",
>         "subsystem_vendor": "ffff",
>         "subsystem_device": "ffff"
>       }
>     ]
>   }
> 
> Get only the required kernel modules for a given driver:
> 
>   ~$ usertools/dpdk-pmdinfo.py build/app/dpdk-testpmd | \
>     jq '.[] | select(.name == "net_i40e").kmod'
>   "* igb_uio | uio_pci_generic | vfio-pci"
> 
> Get only the required kernel modules for a given device:
> 
>   ~$ usertools/dpdk-pmdinfo.py build/app/dpdk-testpmd | \
>     jq '.[] | select(.pci_ids[] | .vendor == "15b3" and .device == "1013").kmod'
>   "* ib_uverbs & mlx5_core & mlx5_ib"
> 
> Print the list of drivers which define multiple parameters without
> string separators:
> 
>   ~$ usertools/dpdk-pmdinfo.py build/app/dpdk-testpmd | \
>     jq '.[] | select(.params!=null and (.params|test("=[^ ]+="))) | {name, params}'
>   ...
> 
> The script passes flake8, black, isort and pylint checks.
> 
> I have tested this with a matrix of python/pyelftools versions:
> 
>                               pyelftools
>                 0.22 0.23 0.24 0.25 0.26 0.27 0.28 0.29
>           3.6     ok   ok   ok   ok   ok   ok   ok   ok
>           3.7     ok   ok   ok   ok   ok   ok   ok   ok
>    Python 3.8     ok   ok   ok   ok   ok   ok   ok   ok
>           3.9     ok   ok   ok   ok   ok   ok   ok   ok
>           3.10  fail fail fail fail   ok   ok   ok   ok
> 
> All failures with python 3.10 are related to the same issue:
> 
>    File "elftools/construct/lib/container.py", line 5, in <module>
>      from collections import MutableMapping
>    ImportError: cannot import name 'MutableMapping' from 'collections'
> 
> Python 3.10 support is only available since pyelftools 0.26. The script
> will only work with Python 3.6 and later. Update the minimal system
> requirements and release notes.
> 
> NB: The output produced by the legacy -r/--raw flag can be obtained with
> the following command:
> 
>    strings build/app/dpdk-testpmd | sed -n 's/^PMD_INFO_STRING= //p'
> 
> Cc: Olivier Matz <olivier.matz at 6wind.com>
> Cc: Ferruh Yigit <ferruh.yigit at xilinx.com>
> Cc: Bruce Richardson <bruce.richardson at intel.com>
> Signed-off-by: Robin Jarry <rjarry at redhat.com>


For 'subsystem_vendor' & 'subsystem_device', the value "ffff" means it 
is not explicitly defined, so it gets default value.
What do you think to omit those as well, when value is "ffff", to reduce 
noise on the output?



BTW, I have detected some duplicates in the output, like [1], [2] & [3]. 
It seems related to the duplicates in the code, cc'ed maintainers.

[1]:
   {
     "name": "net_qede",
     "kmod": "* igb_uio | uio_pci_generic | vfio-pci",
     "pci_ids": [
       {
         "vendor": "1077",
         "device": "1634",
         "subsystem_vendor": "ffff",
         "subsystem_device": "ffff"
       },
       {
         "vendor": "1077",
         "device": "1629",
         "subsystem_vendor": "ffff",
         "subsystem_device": "ffff"
       },
       {
         "vendor": "1077",
         "device": "1634",
         "subsystem_vendor": "ffff",
         "subsystem_device": "ffff"
       },
...

[2]
   {
     "name": "mempool_cnxk",
     "params": "max_pools=<128-1048576>",
     "kmod": "vfio-pci",
     "pci_ids": [
       {
         "vendor": "177d",
         "device": "a0fb",
         "subsystem_vendor": "ffff",
         "subsystem_device": "b900"
       },
       {
         "vendor": "177d",
         "device": "a0fb",
         "subsystem_vendor": "ffff",
         "subsystem_device": "b900"
       },
...

[3]
   {
     "name": "net_cn10k",
     "kmod": "vfio-pci",
     "pci_ids": [
       {
         "vendor": "177d",
         "device": "a063",
         "subsystem_vendor": "ffff",
         "subsystem_device": "b900"
       },
       {
         "vendor": "177d",
         "device": "a063",
         "subsystem_vendor": "ffff",
         "subsystem_device": "b900"
       },
...