Setting up DPDK PMD Test Suite

[Andrew Rybchenko]

Hi Adam,

On 11/16/23 23:03, Adam Hassick wrote:
> Hi Andrew,
>
>     If you use copy of dpdk-ethdev-ts has
>     398e272495143884274f5a53c6fe0cc16df41052, you don't need to pass
>     --trc-tag=pci-8086-1572 any more since corresponding changeset
>     updates expectations to have the same for pci-8086-1583.
>
>
> I'll try this for the next run.
>
>     Sorry, but I've failed to find what's wrong there.
>
>
> That if statement works if using the traditional single-bracket 
> conditional, or it needs to be rewritten as "[[ -z "${test_log}" ]] || 
> [[ ! -r "${test_log}" ]]". The latter is the change I made, but both work.

Thanks a lot. Hopefully fixed.

>
>     As far as I can see LLDP packets spoil testing results:
>     https://ts-factory.io/bublik/v2/log/362398?focusId=362760&mode=treeAndinfoAndlog&experimental=true&lineNumber=1_63
>     <https://ts-factory.io/bublik/v2/log/362398?focusId=362760&mode=treeAndinfoAndlog&experimental=true&lineNumber=1_63>
>
>     As far as I can see main prologue disables FW LLDP on Tester
>     https://ts-factory.io/bublik/v2/log/362398?focusId=362400&mode=treeAndinfoAndlog&experimental=true&lineNumber=1_80
>     <https://ts-factory.io/bublik/v2/log/362398?focusId=362400&mode=treeAndinfoAndlog&experimental=true&lineNumber=1_80>
>     but I guess it could be still enabled on DUT side and DPDK do not
>     provide means to disable it as far as I know. I vaguely remember
>     that Intel provides FW configuration tools which can do it.
>     It is interesting since DPDK gets unexpected LLDP packets but may
>     be packets sent by FW go via loopback and visible to PF as well.
>     Other possible source of LLDP packet is a switch if NICs are
>     connected via switch. If so, LLDP should be disabled on
>     corresponding switch ports.
>
>     As far as I can see fixing the problem should make results much
>     closer. However, I already see some differences in behaviour which
>     should be simply fixed in TRC. For example, X710 gets 9 packets
>     less than configuration number of Rx descriptors, but XL710 gets
>     10 packets less.
>
>
> I have the "disable-fw-lldp" private flag set on both of the XL710 
> ports on the DUT machine. Very strange how there are still LLDP 
> packets appearing in there.

Me too. Corresponding packet has source MAC from Peer/Tester machine NIC.
It is really strange since prologue disabled LLDP there as well. I'll 
try to play with it locally more, but have no good ideas in fact.

> These systems are not connected to any switch, so maybe a service on 
> the DUT itself is sending them. I'm not sure how that could be 
> happening though, because I don't have the LLDP daemon installed on 
> either system.
>
>     Also I see that performance tests are not run because of failed
>     prologue:
>     https://ts-factory.io/bublik/v2/log/362398?focusId=369564&mode=treeAndinfoAndlog&experimental=true
>     <https://ts-factory.io/bublik/v2/log/362398?focusId=369564&mode=treeAndinfoAndlog&experimental=true>
>     I'll investigate it, but I guess the source of difference is that
>     we always run tests on single interface. Just add -p0
>     (--cfg=iol-dts-xl710-p0) to your configuration name. You don't
>     need to change ts-rigs for it since the suffix is handled by
>     generic code. It simply comments the second instance and forces
>     take the first interface only into account. Initially it was
>     introduced to run independent tests on different ports to be able
>     to share configuration, but I guess right now it has limitations
>     for some packages like representors which require entire NIC.
>
>
> I can try that and will see if it works.

This problem is fixed in fresh TE and dpdk-ethdev-ts published on GitHub.

Regards,
Andrew.

>
> Thanks,
> Adam
>
> On Wed, Nov 8, 2023 at 2:20 AM Andrew Rybchenko 
> <andrew.rybchenko at oktetlabs.ru> wrote:
>
>     Hi Adam,
>
>     On 11/7/23 23:30, Adam Hassick wrote:
>>     Hi Andrew,
>>
>>     The runner machine was missing a dependency for one of the
>>     scripts, "pixz". After installing that, it appears to have
>>     worked. I can see the results listed on the ts-factory Bublik
>>     instance.
>
>     If you use copy of dpdk-ethdev-ts has
>     398e272495143884274f5a53c6fe0cc16df41052, you don't need to pass
>     --trc-tag=pci-8086-1572 any more since corresponding changeset
>     updates expectations to have the same for pci-8086-1583.
>
>>     In the latest revision of ts-rigs, there appears to be a syntax
>>     error at line 42 within the script located at
>>     "ts-rigs/scripts/publish_logs/prj/ts-factory/publish", within the
>>     if condition. I fixed it locally to get it to run.
>
>     Sorry, but I've failed to find what's wrong there.
>
>>     Taking a quick look at a comparison against your most recent X710
>>     run, it looks like we're NOK on around ~400 more test cases. By
>>     percentage of tests, we're 1% off, however, it looks like whole
>>     subsets of the test suite that contain low numbers of tests are
>>     failing. I wonder if this is due to differences between the Intel
>>     X710 and XL710 or issues in our dev testbed.
>
>     As far as I can see LLDP packets spoil testing results:
>     https://ts-factory.io/bublik/v2/log/362398?focusId=362760&mode=treeAndinfoAndlog&experimental=true&lineNumber=1_63
>     <https://ts-factory.io/bublik/v2/log/362398?focusId=362760&mode=treeAndinfoAndlog&experimental=true&lineNumber=1_63>
>
>     As far as I can see main prologue disables FW LLDP on Tester
>     https://ts-factory.io/bublik/v2/log/362398?focusId=362400&mode=treeAndinfoAndlog&experimental=true&lineNumber=1_80
>     <https://ts-factory.io/bublik/v2/log/362398?focusId=362400&mode=treeAndinfoAndlog&experimental=true&lineNumber=1_80>
>     but I guess it could be still enabled on DUT side and DPDK do not
>     provide means to disable it as far as I know. I vaguely remember
>     that Intel provides FW configuration tools which can do it.
>     It is interesting since DPDK gets unexpected LLDP packets but may
>     be packets sent by FW go via loopback and visible to PF as well.
>     Other possible source of LLDP packet is a switch if NICs are
>     connected via switch. If so, LLDP should be disabled on
>     corresponding switch ports.
>
>     As far as I can see fixing the problem should make results much
>     closer. However, I already see some differences in behaviour which
>     should be simply fixed in TRC. For example, X710 gets 9 packets
>     less than configuration number of Rx descriptors, but XL710 gets
>     10 packets less.
>
>     Also I see that performance tests are not run because of failed
>     prologue:
>     https://ts-factory.io/bublik/v2/log/362398?focusId=369564&mode=treeAndinfoAndlog&experimental=true
>     <https://ts-factory.io/bublik/v2/log/362398?focusId=369564&mode=treeAndinfoAndlog&experimental=true>
>     I'll investigate it, but I guess the source of difference is that
>     we always run tests on single interface. Just add -p0
>     (--cfg=iol-dts-xl710-p0) to your configuration name. You don't
>     need to change ts-rigs for it since the suffix is handled by
>     generic code. It simply comments the second instance and forces
>     take the first interface only into account. Initially it was
>     introduced to run independent tests on different ports to be able
>     to share configuration, but I guess right now it has limitations
>     for some packages like representors which require entire NIC.
>
>     Regards,
>     Andrew.
>
>>     Thanks,
>>     Adam
>
>     (dropped history, to keep mail size small)
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mails.dpdk.org/archives/ci/attachments/20231120/cea8dd59/attachment.htm>