[dpdk-dev] Issue with net/netvsc pmd in 18.11.9

[Kevin Traynor]

On 16/09/2020 13:00, Dey, Souvik wrote:
> Hi All,
>       I updated from dpdk 18.11.6 to 18.11.9 and my netvsc pmd stopped working as expected. Firstly, I saw a crash in tx packets as soon as the dpdk app comes up. In doing some googling found a bug that was fixed after 18.11.9 https://patches.dpdk.org/patch/75001/ . Ported this fix and was able to get rid of the seg fault, but now stuck at another issue. When we are transmitting packets of size within HN_TXCOPY_THRESHOLD we 

Hi,

The patch mentioned is merged to the 18.11 branch in dpdk-stable repo
and part of 18.11.10-rc1, so expected to be in 18.11.10. ETA 28th September.

In case it's already fixed, you can try the 18.11.10-rc1 release. These
netvsc backports were added since 18.11.9. The corresponding dpdk main
commit id's are in the full log.

$ git log --oneline v18.11.9..HEAD .
363bdf0f49 net/netvsc: fix chimney index
5b3a327709 net/netvsc: fix crash during Tx
77cf88c0c6 net/netvsc: fix underflow when Rx external mbuf
59257aaaa1 net/netvsc: do not spin forever waiting for reply

The list of fixes that were tagged for stable but not backported is
here:
http://inbox.dpdk.org/stable/20200901100115.72365-1-ktraynor@redhat.com/

are all good but any larger packets/fragmented packets are getting
dropped after some time. As soon as we start to receive the transmit
completion event NVS_TYPE_RNDIS_ACK as failed, packets with size greater
than HN_TXCOPY_THRESHOLD starts to drop and never recovers. Packets less
than HN_TXCOPY_THRESHOLD works properly there after though. Any idea why
this is happening and is there is some fix which is already there which
is done after 18.11.9 release ?

Maybe you can bisect the 18.11 branch to find where it stops working,
there is only ~10 netvsc commits between 18.11.6 and 18.11.9. The
release notes [1] or git history will tell you which commits are in
which releases. Bear in mind that some commits were rebased for 18.11,
so if you find the offending commit, you might want to check that it was
correctly backported too.

Kevin.

[1] http://doc.dpdk.org/guides-18.11/rel_notes/release_18_11.html

> We are at the critical part of our release, so any help in this regard will be highly appreciated. Thanks in advance for all the help.
> 
> PS: I also tried to put the net/netvsc: return the correct chimney index <https://patches.dpdk.org/patch/75118/>  fix in but nothing changed.
> 
> --
> Regards,
> Souvik
> 
> Log Snippet:
> I can see the below errors coming after some transmitting.
> hn_xmit_sg() tx: port 2:0 tx 4294967295 segs 2 size 0
> hn_xmit_sg() tx: port 2:0 tx 4294967295 segs 3 size 0
> hn_xmit_sg() tx: port 2:0 tx 4294967295 segs 2 size 0
> hn_xmit_sg() tx: port 2:0 tx 4294967295 segs 3 size 0
> hn_xmit_sg() tx: port 2:0 tx 4294967295 segs 2 size 0
> hn_xmit_sg() tx: port 2:0 tx 4294967295 segs 3 size 0
> hn_xmit_sg() tx: port 2:0 tx 4294967295 segs 2 size 0
> hn_xmit_sg() tx: port 2:0 tx 4294967295 segs 3 size 0
> hn_xmit_sg() tx: port 2:0 tx 4294967295 segs 2 size 0
> hn_xmit_sg() tx: port 2:0 tx 4294967295 segs 3 size 0
> hn_flush_txagg() tx: port 2:0 tx 2048 size 302
> hn_nvs_send_completed() tx: port 2:0 complete tx 4294967295 packets 1 bytes 1514
> hn_nvs_send_completed() tx: port 2:0 complete tx 4294967295 packets 1 bytes 1514
> hn_nvs_send_completed() tx: port 2:0 complete tx 4294967295 packets 1 bytes 1514
> hn_nvs_send_completed() tx: port 2:0 complete tx 4294967295 packets 1 bytes 1514
> hn_nvs_send_completed() tx: port 2:0 complete tx 4294967295 packets 1 bytes 1514
> hn_nvs_send_completed() tx: port 2:0 complete tx 4294967295 packets 1 bytes 1514
> hn_nvs_send_completed() tx: port 2:0 complete tx 4294967295 packets 1 bytes 1514
> hn_nvs_send_completed() tx: port 2:0 complete tx 4294967295 packets 1 bytes 1514
> hn_nvs_send_completed() tx: port 2:0 complete tx 4294967295 packets 1 bytes 1514
> hn_nvs_send_completed() tx: port 2:0 complete tx 4294967295 packets 1 bytes 1514
> hn_nvs_send_completed() tx: port 2:0 complete tx 2048 packets 1 bytes 242
> 
> hn_xmit_sg() tx: port 2:0 tx 4294967295 segs 2 size 0
> hn_xmit_sg() tx: port 2:0 tx 4294967295 segs 2 size 0
> hn_xmit_sg() tx: port 2:0 tx 4294967295 segs 2 size 0
> hn_xmit_sg() tx: port 2:0 tx 4294967295 segs 3 size 0
> hn_xmit_sg() tx: port 2:0 tx 4294967295 segs 2 size 0
> hn_xmit_sg() tx: port 2:0 tx 4294967295 segs 3 size 0
> hn_xmit_sg() tx: port 2:0 tx 4294967295 segs 2 size 0
> hn_xmit_sg() tx: port 2:0 tx 4294967295 segs 2 size 0
> hn_xmit_sg() tx: port 2:0 tx 4294967295 segs 3 size 0
> hn_xmit_sg() tx: port 2:0 tx 4294967295 segs 2 size 0
> hn_flush_txagg() tx: port 2:0 tx 2624 size 302
> hn_nvs_send_completed() tx: port 2:0 complete tx 4294967295 failed status 2
> hn_nvs_send_completed() tx: port 2:0 complete tx 4294967295 failed status 2
> hn_nvs_send_completed() tx: port 2:0 complete tx 4294967295 failed status 2
> hn_nvs_send_completed() tx: port 2:0 complete tx 4294967295 failed status 2
> hn_nvs_send_completed() tx: port 2:0 complete tx 4294967295 packets 1 bytes 1514
> hn_nvs_send_completed() tx: port 2:0 complete tx 4294967295 packets 1 bytes 1514
> hn_nvs_send_completed() tx: port 2:0 complete tx 4294967295 packets 1 bytes 1514
> hn_nvs_send_completed() tx: port 2:0 complete tx 4294967295 packets 1 bytes 1514
> hn_nvs_send_completed() tx: port 2:0 complete tx 4294967295 packets 1 bytes 1514
> hn_nvs_send_completed() tx: port 2:0 complete tx 4294967295 packets 1 bytes 1514
> hn_nvs_send_completed() tx: port 2:0 complete tx 2624 packets 1 bytes 242
> 
> hn_xmit_sg() tx: port 2:0 tx 4294967295 segs 2 size 0
> hn_xmit_sg() tx: port 2:0 tx 4294967295 segs 2 size 0
> hn_xmit_sg() tx: port 2:0 tx 4294967295 segs 2 size 0
> hn_xmit_sg() tx: port 2:0 tx 4294967295 segs 3 size 0
> hn_xmit_sg() tx: port 2:0 tx 4294967295 segs 2 size 0
> hn_xmit_sg() tx: port 2:0 tx 4294967295 segs 3 size 0
> hn_xmit_sg() tx: port 2:0 tx 4294967295 segs 2 size 0
> hn_xmit_sg() tx: port 2:0 tx 4294967295 segs 2 size 0
> hn_xmit_sg() tx: port 2:0 tx 4294967295 segs 3 size 0
> hn_xmit_sg() tx: port 2:0 tx 4294967295 segs 2 size 0
> hn_flush_txagg() tx: port 2:0 tx 2688 size 302
> hn_nvs_send_completed() tx: port 2:0 complete tx 4294967295 failed status 2
> hn_nvs_send_completed() tx: port 2:0 complete tx 4294967295 failed status 2
> hn_nvs_send_completed() tx: port 2:0 complete tx 4294967295 failed status 2
> hn_nvs_send_completed() tx: port 2:0 complete tx 4294967295 failed status 2
> hn_nvs_send_completed() tx: port 2:0 complete tx 4294967295 failed status 2
> hn_nvs_send_completed() tx: port 2:0 complete tx 4294967295 failed status 2
> hn_nvs_send_completed() tx: port 2:0 complete tx 4294967295 failed status 2
> hn_nvs_send_completed() tx: port 2:0 complete tx 4294967295 failed status 2
> hn_nvs_send_completed() tx: port 2:0 complete tx 4294967295 failed status 2
> hn_nvs_send_completed() tx: port 2:0 complete tx 4294967295 failed status 2
> hn_nvs_send_completed() tx: port 2:0 complete tx 2688 packets 1 bytes 242
> hn_flush_txagg() tx: port 2:0 tx 3264 size 571
> hn_nvs_send_completed() tx: port 2:0 complete tx 3264 packets 1 bytes 511
> hn_rxpkt() rx: port 2:0 RX id 3 size 511 type 0x11 ol_flags 0x2
> hn_flush_txagg() tx: port 2:0 tx 3328 size 571
> hn_nvs_send_completed() tx: port 2:0 complete tx 3328 packets 1 bytes 511
> hn_rxpkt() rx: port 2:0 RX id 3 size 512 type 0x11 ol_flags 0x2
> hn_xmit_sg() tx: port 2:0 tx 4294967295 segs 2 size 0
> hn_nvs_send_completed() tx: port 2:0 complete tx 4294967295 failed status 2
> hn_rxpkt() rx: port 2:0 RX id 3 size 512 type 0x11 ol_flags 0x2
> hn_xmit_sg() tx: port 2:0 tx 4294967295 segs 2 size 0
> hn_nvs_send_completed() tx: port 2:0 complete tx 4294967295 failed status 2
> hn_flush_txagg() tx: port 2:0 tx 3392 size 120
> hn_nvs_send_completed() tx: port 2:0 complete tx 3392 packets 1 bytes 60
> hn_rxpkt() rx: port 2:0 RX id 3 size 42 type 0x1 ol_flags 0
>