[dpdk-users] mlx5 pmd + rdma-core 28 init failure

[Benoit Ganne (bganne)]

Adding MLX5 PMD maintainers.

I also checked with latest rdma-core master and latest DPDK master and it is failing with the same issue.
Any recommendation?

Best
ben

> -----Original Message-----
> From: Benoit Ganne (bganne)
> Sent: mercredi 1 avril 2020 18:52
> To: users at dpdk.org
> Subject: mlx5 pmd + rdma-core 28 init failure
> 
> Hi all,
> 
> I have troubles making DPDK v20.02 MLX5 PMD working with rdma-core v28.0:
> it looks like the flow initialization done while initializing the device
> fails in rdma-core providers/mlx5/dr_table.c:mlx5dv_dr_table_create()
> because of unsupported parameters.
> The issue is the following test in rdma-core
> providers/mlx5/dr_table.c:mlx5dv_dr_table_create():
>     if (level && !dmn->info.supp_sw_steering) {
>         errno = EOPNOTSUPP;
>         goto dec_ref;
>     }
> Where level == 65534 and dmn->info.supp_sw_steering == false, hence the
> test is true and the function returns EOPNOTSUPP.
> 
> Is this expected? It does work fine with ibv instead of dv API. Any help
> appreciated.
> 
> Here is what I do:
>    1) checkout & compile rdma-core v28.0
> ~# git clone https://github.com/linux-rdma/rdma-core
> ~# cd rdma-core
> ~# git checkout v28.0
> ~# mkdir build
> ~# cd build
> ~# CFLAGS=-fPIC cmake -DIN_PLACE=1 -DENABLE_STATIC=1 -GNinja ..
> ~# ninja
> 
>    2) checkout & compile dpdk v20.02
> ~# git clone git://dpdk.org/dpdk
> ~# cd dpdk
> ~# make config T=x86_64-native-linuxapp-gcc
> ~# sed -ri 's,(MLX5_PMD=).*,\1y,' build/.config
> ~# sed -ri 's,(IBVERBS_LINK_STATIC_PMD=).*,\1y,' build/.config
> ~# make EXTRA_CFLAGS=-I/home/bganne/src/rdma-core/build/include
> EXTRA_LDFLAGS=-L/home/bganne/src/rdma-core/build/lib
> PKG_CONFIG_PATH=/home/bganne/src/rdma-core/build/lib/pkgconfig
> ~# sudo gdb --args ./build/app/testpmd -w 0000:5e:00.0 -w 0000:5e:00.1 -l
> 4,11,35 -- -a --forward-mode=rxonly -i
> 
> The backtrace looks like this:
> #0  mlx5dv_dr_table_create (dmn=0x555556c641b0, level=65534) at
> ../providers/mlx5/dr_table.c:183
> #1  0x0000555555dfaeaa in flow_dv_tbl_resource_get (dev=<optimized out>,
> table_id=65534, egress=<optimized out>, transfer=<optimized out>,
> error=0x7fffffffdca0) at
> /home/bganne/src/dpdk/drivers/net/mlx5/mlx5_flow_dv.c:6746
> #2  0x0000555555e02b28 in __flow_dv_translate
> (dev=dev at entry=0x555556bbcdc0 <rte_eth_devices>, dev_flow=0x100388300,
> attr=<optimized out>, items=<optimized out>, actions=<optimized out>,
> error=<optimized out>) at
> /home/bganne/src/dpdk/drivers/net/mlx5/mlx5_flow_dv.c:7503
> #3  0x0000555555e04954 in flow_dv_translate (dev=0x555556bbcdc0
> <rte_eth_devices>, dev_flow=<optimized out>, attr=<optimized out>,
> items=<optimized out>, actions=<optimized out>, error=<optimized out>) at
> /home/bganne/src/dpdk/drivers/net/mlx5/mlx5_flow_dv.c:8841
> #4  0x0000555555df152f in flow_drv_translate (error=0x7fffffffdca0,
> actions=0x7fffffffdce0, items=0x7fffffffdcc0, attr=0x7fffffffbb88,
> dev_flow=<optimized out>, dev=0x555556bbcdc0 <rte_eth_devices>) at
> /home/bganne/src/dpdk/drivers/net/mlx5/mlx5_flow.c:2571
> #5  flow_create_split_inner (error=0x7fffffffdca0, external=false,
> actions=0x7fffffffdce0, items=0x7fffffffdcc0, attr=0x7fffffffbb88,
> prefix_layers=0, sub_flow=0x0, flow=0x1003885c0, dev=0x555556bbcdc0
> <rte_eth_devices>) at
> /home/bganne/src/dpdk/drivers/net/mlx5/mlx5_flow.c:3490
> #6  flow_create_split_metadata (error=0x7fffffffdca0, external=false,
> actions=0x7fffffffdce0, items=0x7fffffffdcc0, attr=0x7fffffffbb88,
> prefix_layers=0, flow=0x1003885c0, dev=0x555556bbcdc0 <rte_eth_devices>)
> at /home/bganne/src/dpdk/drivers/net/mlx5/mlx5_flow.c:3865
> #7  flow_create_split_meter (error=0x7fffffffdca0, external=false,
> actions=0x7fffffffdce0, items=<optimized out>, attr=0x7fffffffdc94,
> flow=0x1003885c0, dev=0x555556bbcdc0 <rte_eth_devices>) at
> /home/bganne/src/dpdk/drivers/net/mlx5/mlx5_flow.c:4121
> #8  flow_create_split_outer (error=0x7fffffffdca0, external=false,
> actions=0x7fffffffdce0, items=<optimized out>, attr=0x7fffffffdc94,
> flow=0x1003885c0, dev=0x555556bbcdc0 <rte_eth_devices>) at
> /home/bganne/src/dpdk/drivers/net/mlx5/mlx5_flow.c:4178
> #9  flow_list_create (dev=dev at entry=0x555556bbcdc0 <rte_eth_devices>,
> list=list at entry=0x0, attr=attr at entry=0x7fffffffdc94,
> items=items at entry=0x7fffffffdcc0, actions=actions at entry=0x7fffffffdce0,
> external=external at entry=false, error=0x7fffffffdca0) at
> /home/bganne/src/dpdk/drivers/net/mlx5/mlx5_flow.c:4306
> #10 0x0000555555df8587 in mlx5_flow_discover_mreg_c
> (dev=dev at entry=0x555556bbcdc0 <rte_eth_devices>) at
> /home/bganne/src/dpdk/drivers/net/mlx5/mlx5_flow.c:5747
> #11 0x0000555555d692a6 in mlx5_dev_spawn (config=..., spawn=0x1003e9e00,
> dpdk_dev=0x555556dd6fe0) at
> /home/bganne/src/dpdk/drivers/net/mlx5/mlx5.c:2763
> #12 mlx5_pci_probe (pci_drv=<optimized out>, pci_dev=<optimized out>) at
> /home/bganne/src/dpdk/drivers/net/mlx5/mlx5.c:3363
> #13 0x0000555555a411c8 in pci_probe_all_drivers ()
> #14 0x0000555555a412f8 in rte_pci_probe ()
> #15 0x0000555555a083da in rte_bus_probe ()
> #16 0x00005555559f204d in rte_eal_init ()
> #17 0x00005555556a0d45 in main ()
> 
> Best
> ben