[dpdk-users] Unregistered mempool in secondary

[Shahaf Shuler]

Only w/ Mellanox NICs or in general?

From: Tom Barbette <barbette at kth.se>
Sent: Thursday, December 6, 2018 11:38 AM
To: Shahaf Shuler <shahafs at mellanox.com>; Yongseok Koh <yskoh at mellanox.com>
Cc: users at dpdk.org
Subject: RE: Unregistered mempool in secondary


I'm trying to reproduce with testpmd. But when I launch :

sudo ~/dpdk/x86_64-native-linuxapp-gcc/app/testpmd -w "0000:03:00.0" -w "0000:82:00.0" -- -i

sudo ~/dpdk/x86_64-native-linuxapp-gcc/app/testpmd -w "0000:03:00.0" -w "0000:82:00.0" --proc-type=secondary​ -- -i



I get on the secondary:
EAL: Error - exiting with code: 1

Cause: Creation of mbuf pool for socket 0 failed: File exists

I also had that with 18.11.

Tom



________________________________
De : Shahaf Shuler <shahafs at mellanox.com<mailto:shahafs at mellanox.com>>
Envoyé : jeudi 6 décembre 2018 10:21
À : Tom Barbette; Yongseok Koh
Cc : users at dpdk.org<mailto:users at dpdk.org>
Objet : RE: Unregistered mempool in secondary

Do you reproduce this issue w/ testpmd?
Can you provide the instruction on how to?

From: Tom Barbette <barbette at kth.se<mailto:barbette at kth.se>>
Sent: Thursday, December 6, 2018 10:19 AM
To: Shahaf Shuler <shahafs at mellanox.com<mailto:shahafs at mellanox.com>>; Yongseok Koh <yskoh at mellanox.com<mailto:yskoh at mellanox.com>>
Cc: users at dpdk.org<mailto:users at dpdk.org>
Subject: RE: Unregistered mempool in secondary


Hi Shahaf,



Both devices are initialized in the primary with rte_eth_dev_start, and queues also (each with their own packet pool). It's just that the secondary read from the initialized queue.



If you're mentionning something more to do, then I did not understand. Which function would that be? To ask port 0 to register port 1's pools and vice versa?



Thanks !



Tom



________________________________
De : Shahaf Shuler <shahafs at mellanox.com<mailto:shahafs at mellanox.com>>
Envoyé : jeudi 6 décembre 2018 08:18
À : Tom Barbette; Yongseok Koh
Cc : users at dpdk.org<mailto:users at dpdk.org>
Objet : RE: Unregistered mempool in secondary

Hi Tom,

There is a complete isolation between the resources of each port.
Meaning, mempool cannot be registered on one port and be used on the other (w/o explicit registration).

In order your solution to work, the primary process needs to probe both ports, so that the mempool_walk will happen on both and the pool will be registered to both.

From: Tom Barbette <barbette at kth.se<mailto:barbette at kth.se>>
Sent: Thursday, December 6, 2018 1:41 AM
To: Shahaf Shuler <shahafs at mellanox.com<mailto:shahafs at mellanox.com>>; Yongseok Koh <yskoh at mellanox.com<mailto:yskoh at mellanox.com>>
Cc: users at dpdk.org<mailto:users at dpdk.org>
Subject: Unregistered mempool in secondary

Hi mlx5 maintainers,

Since we're using a second ConnectX 5 NIC plugged to our second CPU socket, we see the following message when flowing packets through that port :

net_mlx5: port 1 using address (0x6000712742c0) of unregistered mempool in secondary process, please create mempool before rte_eth_dev_start()

The secondary process has all of the primary process pools registered via rte_mempool_walk (2 as we have 2 numa nodes). But that address in the error message is actually not between any of the pool->mz->addr_64  + pool->mz->size range. So maybe this comes from an automatically created pool of the primary ? If so, how can we force it to be created before we call eth_dev_start?

We think the only change is that the second port is now a dedicated NIC on the second socket, instead of the second port of the same NIC on the first socket. But we are not 100% sure this is the triggering event. The same test worked before, still with DPDK 18.08.

Thanks,
Tom