[PATCH v4] usertools: rewrite pmdinfo

[Ferruh Yigit]

On 9/20/2022 6:48 PM, Ferruh Yigit wrote:
> On 9/20/2022 11:42 AM, Robin Jarry wrote:
> 
>>
>> dpdk-pmdinfo.py does not produce any parseable output. The -r/--raw flag
>> merely prints multiple independent JSON lines which cannot be fed
>> directly to any JSON parser. Moreover, the script complexity is rather
>> high for such a simple task: extracting PMD_INFO_STRING from .rodata ELF
>> sections. Rewrite it so that it can produce valid JSON.
>>
>> Remove the PCI database parsing for PCI-ID to Vendor-Device names
>> conversion. This should be done by external scripts (if really needed).
>>
>> Here are some examples of use with jq:
>>
>> Get the complete info for a given driver:
>>
>>   ~$ usertools/dpdk-pmdinfo.py build/app/dpdk-testpmd | \
>>     jq '.[] | select(.name == "dmadev_idxd_pci")'
>>   {
>>     "name": "dmadev_idxd_pci",
>>     "params": "max_queues=0",
>>     "kmod": "vfio-pci",
>>     "pci_ids": [
>>       {
>>         "vendor": "8086",
>>         "device": "0b25"
>>       }
>>     ]
>>   }
>>
>> Get only the required kernel modules for a given driver:
>>
>>   ~$ usertools/dpdk-pmdinfo.py build/app/dpdk-testpmd | \
>>     jq '.[] | select(.name == "net_i40e").kmod'
>>   "* igb_uio | uio_pci_generic | vfio-pci"
>>
>> Get only the required kernel modules for a given device:
>>
>>   ~$ usertools/dpdk-pmdinfo.py build/app/dpdk-testpmd | \
>>     jq '.[] | select(.pci_ids[] | .vendor == "15b3" and .device == 
>> "1013").kmod'
>>   "* ib_uverbs & mlx5_core & mlx5_ib"
>>
>> Print the list of drivers which define multiple parameters without
>> space separators:
>>
>>   ~$ usertools/dpdk-pmdinfo.py build/app/dpdk-testpmd | \
>>     jq '.[] | select(.params!=null and (.params|test("=[^ ]+="))) | 
>> {name, params}'
>>   ...
>>
>> The script passes flake8, black, isort and pylint checks.
>>
>> I have tested this with a matrix of python/pyelftools versions:
>>
>>                               pyelftools
>>                 0.22 0.23 0.24 0.25 0.26 0.27 0.28 0.29
>>           3.6     ok   ok   ok   ok   ok   ok   ok   ok
>>           3.7     ok   ok   ok   ok   ok   ok   ok   ok
>>    Python 3.8     ok   ok   ok   ok   ok   ok   ok   ok
>>           3.9     ok   ok   ok   ok   ok   ok   ok   ok
>>           3.10  fail fail fail fail   ok   ok   ok   ok
>>
>> All failures with python 3.10 are related to the same issue:
>>
>>    File "elftools/construct/lib/container.py", line 5, in <module>
>>      from collections import MutableMapping
>>    ImportError: cannot import name 'MutableMapping' from 'collections'
>>
>> Python 3.10 support is only available since pyelftools 0.26. The script
>> will only work with Python 3.6 and later. Update the minimal system
>> requirements and release notes.
>>
>> NB: The output produced by the legacy -r/--raw flag can be obtained with
>> the following command:
>>
>>    strings build/app/dpdk-testpmd | sed -n 's/^PMD_INFO_STRING= //p'
>>
>> Cc: Olivier Matz <olivier.matz at 6wind.com>
>> Cc: Ferruh Yigit <ferruh.yigit at xilinx.com>
>> Cc: Bruce Richardson <bruce.richardson at intel.com>
>> Signed-off-by: Robin Jarry <rjarry at redhat.com>
> 
> <...>
> 
>> diff --git a/doc/guides/rel_notes/release_22_11.rst 
>> b/doc/guides/rel_notes/release_22_11.rst
>> index 8c021cf0505e..67054f5acdc9 100644
>> --- a/doc/guides/rel_notes/release_22_11.rst
>> +++ b/doc/guides/rel_notes/release_22_11.rst
>> @@ -84,6 +84,11 @@ API Changes
>>      Also, make sure to start the actual text at the margin.
>>      =======================================================
>>
>> +* The ``dpdk-pmdinfo.py`` script was rewritten to produce valid JSON 
>> only.
>> +  PCI-IDs parsing has been removed.
>> +  To get a similar output to the (now removed) ``-r/--raw`` flag, you 
>> may use the following command::
>> +
>> +     strings $dpdk_binary_or_driver | sed -n 's/^PMD_INFO_STRING= //p'
>>
> 
> Empty line is missing (in case there will be a new version for some 
> other reason).
> 
> 
> Thanks for the update,
> Tested-by: Ferruh Yigit <ferruh.yigit at xilinx.com>
> 

Thomas, John,

Should we have documentation for usertools, since they are user facing, 
what do you think?
Can it be possible to find resource for it?

Thanks,
ferruh