[dpdk-dev] mlx5 and secondary processes

[Olivier Gournet]

Hi,

I can't get TX on secondary process to works with the lastest dpdk, it was running fine with dpdk-16.11.
RX/TX is ok on primary processs, and I'm not interested in RX on secondary process. Each process has its
owns TX queues.

I upgraded everything to:
   linux 4.14
   dpdk 17.11-rc4
   rdma-core from yesterday git-master
   mlx-fw 12.21.1000

On seconday process, TX queue gets full and is never emptied.
It seems like bf_reg is correctly re-mapped from secondary process; here's dmesg from primary process:

[...]
[85361.895778] mlx5_1:uar_mmap:1722:(pid 9533): uar idx 0x6, pfn 0x00000000000f8020
[85361.895781] mlx5_1:uar_mmap:1735:(pid 9533): mapped best effort WC at 0x7ffff7faa000, PA 0x00000000f8020000
[85361.895784] mlx5_1:uar_mmap:1722:(pid 9533): uar idx 0x7, pfn 0x00000000000f8021
[85361.895787] mlx5_1:uar_mmap:1735:(pid 9533): mapped best effort WC at 0x7ffff7fa9000, PA 0x00000000f8021000

Then from secondary process:

[...]
[85408.038295] mlx5_1:mlx5_ib_mmap:1778:(pid 9551): mapped internal timer at 0x7ffff7f7f000, PA 0xf8001000
[85408.040229] mlx5_1:uar_mmap:1722:(pid 9551): uar idx 0x6, pfn 0x00000000000f8020
[85408.040233] mlx5_1:uar_mmap:1735:(pid 9551): mapped best effort WC at 0x7ffff7faa000, PA 0x00000000f8020000
[85408.040239] mlx5_1:uar_mmap:1722:(pid 9551): uar idx 0x7, pfn 0x00000000000f8021
[85408.040241] mlx5_1:uar_mmap:1735:(pid 9551): mapped best effort WC at 0x7ffff7fa9000, PA 0x00000000f8021000

But then from mlx5_rxtx.h:mlx5_tx_dbrec_cond_wmb() it doesn't seems like writing to txq->bf_reg has any effect.
I know it isn't supposed to be read, but when reading from primary process is always give *dst==0xe5ccdabae5ccdaba,
whereas from secondary process *dst is zero or the last written value.

I don't really have any other clues, and don't know where to search. Can anybody give me some hints ?

Regards,


=============

# ibv_devinfo -d mlx5_1
hca_id: mlx5_1
         transport:                      InfiniBand (0)
         fw_ver:                         12.21.1000
         node_guid:                      248a:0703:008a:638e
         sys_image_guid:                 248a:0703:008a:638e
         vendor_id:                      0x02c9
         vendor_part_id:                 4115
         hw_ver:                         0x0
         board_id:                       MT_2140110033
         phys_port_cnt:                  1
         Device ports:
                 port:   1
                         state:                  PORT_ACTIVE (4)
                         max_mtu:                4096 (5)
                         active_mtu:             1024 (3)
                         sm_lid:                 0
                         port_lid:               0
                         port_lmc:               0x00
                         link_layer:             Ethernet