[dpdk-dev] Sync up status for Mellanox PMD barrier investigation

Phil Yang (Arm Technology China) Phil.Yang at arm.com
Wed Aug 21 11:57:54 CEST 2019

Some update for this thread.

In the most critical datapath of mlx5 PMD, there are some rte_cio_w/rmb, 'dmb osh' on aarch64, in use.
C11 atomic is good for replacing the rte_smp_r/wmb to relax the data synchronization barrier between CPUs.
However, mlx5 PMD needs to write data back to the  HW, so it used a lot of rte_cio_r/wmb to synchronize data.

Please check details below. All comments are welcomed. Thanks.

//////////////////// Data path ///////////////////////////
drivers/net/mlx5/mlx5_rxtx.c=950=mlx5_rx_err_handle(struct mlx5_rxq_data *rxq, uint8_t mbuf_prepare)
drivers/net/mlx5/mlx5_rxtx.c:1002:           rte_cio_wmb();
drivers/net/mlx5/mlx5_rxtx.c:1004:           rte_cio_wmb();
drivers/net/mlx5/mlx5_rxtx.c:1010:           rte_cio_wmb();
drivers/net/mlx5/mlx5_rxtx.c=1272=mlx5_rx_burst(void *dpdk_rxq, struct rte_mbuf **pkts, uint16_t pkts_n)
drivers/net/mlx5/mlx5_rxtx.c:1385:   rte_cio_wmb();
drivers/net/mlx5/mlx5_rxtx.c:1387:   rte_cio_wmb();
drivers/net/mlx5/mlx5_rxtx.c=1549=mlx5_rx_burst_mprq(void *dpdk_rxq, struct rte_mbuf **pkts, uint16_t pkts_n)
drivers/net/mlx5/mlx5_rxtx.c:1741:   rte_cio_wmb();
drivers/net/mlx5/mlx5_rxtx.c:1745:       rte_cio_wmb();
drivers/net/mlx5/mlx5_rxtx_vec_neon.h=366=rxq_burst_v(struct mlx5_rxq_data *rxq, struct rte_mbuf **pkts, uint16_t pkts_n,
drivers/net/mlx5/mlx5_rxtx_vec_neon.h:530:       rte_cio_rmb();

Commit messages:
net/mlx5: cleanup memory barriers: mlx5_rx_burst

net/mlx5: add Multi-Packet Rx support : mlx5_rx_burst_mprq

net/mlx5: use coherent I/O memory barrier

net/mlx5: extend Rx completion with error handling

net/mlx5: fix synchronization on polling Rx completions

Phil Yang

From: Phil Yang (Arm Technology China)
Sent: Thursday, August 15, 2019 6:35 PM
To: Honnappa Nagarahalli <Honnappa.Nagarahalli at arm.com>
Subject: Sync up status for Mellanox PMD barrier investigation

Hi Honnappa,

I have checked all the barriers in mlx5 PMD data path. In my understanding, it used the barrier correctly (Use DMB to synchronize the memory data between CPUs).
The attachment is the list of positions of these barriers.
I just want to sync up with you the status. Do you have any idea or suggestion on which part should we start to optimization?

Best Regards,
Phil Yang

More information about the dev mailing list