[dpdk-dev] mlx5 and secondary processes

[Xueming(Steven) Li]

Hi Olivier,

Your issue looks like an issue fixed recently:
http://www.dpdk.org/dev/patchwork/patch/34477/

Best Regards,
Xueming

> -----Original Message-----
> From: dev [mailto:dev-bounces at dpdk.org] On Behalf Of Olivier Gournet
> Sent: Tuesday, November 14, 2017 5:58 PM
> To: dev at dpdk.org
> Subject: [dpdk-dev] mlx5 and secondary processes
> 
> Hi,
> 
> I can't get TX on secondary process to works with the lastest dpdk, it was
> running fine with dpdk-16.11.
> RX/TX is ok on primary processs, and I'm not interested in RX on secondary
> process. Each process has its owns TX queues.
> 
> I upgraded everything to:
>    linux 4.14
>    dpdk 17.11-rc4
>    rdma-core from yesterday git-master
>    mlx-fw 12.21.1000
> 
> On seconday process, TX queue gets full and is never emptied.
> It seems like bf_reg is correctly re-mapped from secondary process; here's
> dmesg from primary process:
> 
> [...]
> [85361.895778] mlx5_1:uar_mmap:1722:(pid 9533): uar idx 0x6, pfn
> 0x00000000000f8020 [85361.895781] mlx5_1:uar_mmap:1735:(pid 9533): mapped
> best effort WC at 0x7ffff7faa000, PA 0x00000000f8020000 [85361.895784]
> mlx5_1:uar_mmap:1722:(pid 9533): uar idx 0x7, pfn 0x00000000000f8021
> [85361.895787] mlx5_1:uar_mmap:1735:(pid 9533): mapped best effort WC at
> 0x7ffff7fa9000, PA 0x00000000f8021000
> 
> Then from secondary process:
> 
> [...]
> [85408.038295] mlx5_1:mlx5_ib_mmap:1778:(pid 9551): mapped internal timer
> at 0x7ffff7f7f000, PA 0xf8001000 [85408.040229] mlx5_1:uar_mmap:1722:(pid
> 9551): uar idx 0x6, pfn 0x00000000000f8020 [85408.040233]
> mlx5_1:uar_mmap:1735:(pid 9551): mapped best effort WC at 0x7ffff7faa000,
> PA 0x00000000f8020000 [85408.040239] mlx5_1:uar_mmap:1722:(pid 9551): uar
> idx 0x7, pfn 0x00000000000f8021 [85408.040241] mlx5_1:uar_mmap:1735:(pid
> 9551): mapped best effort WC at 0x7ffff7fa9000, PA 0x00000000f8021000
> 
> But then from mlx5_rxtx.h:mlx5_tx_dbrec_cond_wmb() it doesn't seems like
> writing to txq->bf_reg has any effect.
> I know it isn't supposed to be read, but when reading from primary process
> is always give *dst==0xe5ccdabae5ccdaba, whereas from secondary process
> *dst is zero or the last written value.
> 
> I don't really have any other clues, and don't know where to search. Can
> anybody give me some hints ?
> 
> Regards,
> 
> 
> =============
> 
> # ibv_devinfo -d mlx5_1
> hca_id: mlx5_1
>          transport:                      InfiniBand (0)
>          fw_ver:                         12.21.1000
>          node_guid:                      248a:0703:008a:638e
>          sys_image_guid:                 248a:0703:008a:638e
>          vendor_id:                      0x02c9
>          vendor_part_id:                 4115
>          hw_ver:                         0x0
>          board_id:                       MT_2140110033
>          phys_port_cnt:                  1
>          Device ports:
>                  port:   1
>                          state:                  PORT_ACTIVE (4)
>                          max_mtu:                4096 (5)
>                          active_mtu:             1024 (3)
>                          sm_lid:                 0
>                          port_lid:               0
>                          port_lmc:               0x00
>                          link_layer:             Ethernet