[dpdk-users] mlx5 pmd + rdma-core 28 init failure

[Benoit Ganne (bganne)]

Hi Matan,

> Please let us know if working with devargs dv_flow_en=0 solves the
> issue....
> ./build/app/testpmd --log-level=8 --log-level=pmd.common.mlx5:8 - -log-
> level=pmd.net.mlx5:8 -w 0000:5e:00.0.dv_flow_en=0 -w
> 0000:5e:00.1,dv_flow_en=0 -l 4,11,35 -- -a -- forward-mode=rxonly

Yes it does.
However it seems to let the device in an unstable state: it doesn't stop properly, and I need to restart the whole server for the interface to come back (reloading all mlx drivers modules and/or PCIe resetting is not enough). Here is what happens when I try to start testpmd again after the 1st try:
~# sudo ./build/app/testpmd --log-level=8 --log-level=pmd.common.mlx5:8 --log-level=pmd.net.mlx5:8 -w 0000:5e:00.0,dv_flow_en=0 -w 0000:5e:00.1,dv_flow_en=0 -l 4,11,35 -- -a --forward-mode=rxonly
EAL: Detected 48 lcore(s)
EAL: Detected 2 NUMA nodes
EAL: Multi-process socket /var/run/dpdk/rte/mp_socket
EAL: Selected IOVA mode 'VA'
EAL: No available hugepages reported in hugepages-1048576kB
EAL: Probing VFIO support...
EAL: VFIO support initialized
EAL: PCI device 0000:5e:00.0 on NUMA socket 0
EAL:   probe driver: 15b3:1013 net_mlx5
net_mlx5: mlx5.c:3040: mlx5_pci_probe(): checking device "mlx5_1"
net_mlx5: mlx5.c:3084: mlx5_pci_probe(): no Verbs device matches PCI device 0000:5e:00.0, are kernel drivers loaded?
EAL: Requested device 0000:5e:00.0 cannot be used
EAL: PCI device 0000:5e:00.1 on NUMA socket 0
EAL:   probe driver: 15b3:1013 net_mlx5
net_mlx5: mlx5.c:3040: mlx5_pci_probe(): checking device "mlx5_1"
net_mlx5: mlx5.c:3074: mlx5_pci_probe(): PCI information matches for device "mlx5_1"
net_mlx5: mlx5.c:3305: mlx5_pci_probe(): no E-Switch support detected
net_mlx5: mlx5.c:2178: mlx5_dev_spawn(): naming Ethernet device "0000:5e:00.1"
net_mlx5: mlx5.c:548: mlx5_alloc_shared_ibctx(): DevX is NOT supported
PANIC in mlx5_mr_btree_init():
line 197        assert "!bt->table && !bt->size" failed
14: [./build/app/testpmd(_start+0x2a) [0x560a46a469aa]]
13: [/lib/x86_64-linux-gnu/libc.so.6(__libc_start_main+0xe7) [0x7f082ac3ab97]]
12: [./build/app/testpmd(main+0x96) [0x560a46a50da9]]
11: [./build/app/testpmd(rte_eal_init+0x99d) [0x560a46d6f305]]
10: [./build/app/testpmd(rte_bus_probe+0x4c) [0x560a46d898ae]]
9: [./build/app/testpmd(rte_pci_probe+0x96) [0x560a46dcdc0e]]
8: [./build/app/testpmd(+0x4c4b42) [0x560a46dcdb42]]
7: [./build/app/testpmd(+0x4c4952) [0x560a46dcd952]]
6: [./build/app/testpmd(+0xb33efe) [0x560a4743cefe]]
5: [./build/app/testpmd(+0xb2ff4e) [0x560a47438f4e]]
4: [./build/app/testpmd(+0xb2bd88) [0x560a47434d88]]
3: [./build/app/testpmd(mlx5_mr_btree_init+0x90) [0x560a47b7de13]]
2: [./build/app/testpmd(__rte_panic+0xed) [0x560a46d7e046]]
1: [./build/app/testpmd(rte_dump_stack+0x39) [0x560a46d7dea7]]

> Please let us know also your FW version...

~# sudo mstflint -d 5e:00.0 q
Image type:            FS3
FW Version:            12.26.4012
FW Release Date:       10.12.2019
Product Version:       6.4012
Rom Info:              type=UEFI version=14.19.17 cpu=AMD64,AARCH64
                       type=PXE version=3.5.805 cpu=AMD64
Description:           UID                GuidsNumber
Base GUID:             248a0703005b1414        4
Base MAC:              0000248a075b1414        4
Image VSD:             N/A
Device VSD:            N/A
PSID:                  MT_2150110033
Security Attributes:   N/A

Best,
ben