[dpdk-users] net_mlx5: priv allocation failure

[Adrien Mazarguil]

On Mon, Apr 23, 2018 at 06:03:49PM +0800, Tao Peng wrote:
> Hi ,
> 
> When I run the dpdk l2fwd , I encountered some errors, please help, thanks!
> :)
> 
> $./build/l2fwd -c 0x1 -n 4 -- -p 0x1

Can you confirm this problem only occurs when attempting to run l2fwd as an
unprivileged user?

I just want to make sure, because it's is currently undocumented, untested
and therefore unsupported. This being said, please see below.

> EAL: Detected 64 lcore(s)
> 
> EAL: No free hugepages reported in hugepages-1048576kB
> 
> EAL: Probing VFIO support...
> 
> EAL: WARNING: Master core has no memory on local socket!
> 
> EAL: PCI device 0000:01:00.0 on NUMA socket 0
> 
> EAL:   probe driver: 15b3:1015 net_mlx5
> 
> PMD: net_mlx5: PCI information matches, using device "mlx5_0" (SR-IOV:
> false, MPS: true)
> 
> PMD: net_mlx5: 1 port(s) detected
> 
> PMD: net_mlx5: priv allocation failure
> 
> EAL: Requested device 0000:01:00.0 cannot be used
> 
> EAL: PCI device 0000:01:00.1 on NUMA socket 0
> 
> EAL:   probe driver: 15b3:1015 net_mlx5
> 
> PMD: net_mlx5: PCI information matches, using device "mlx5_1" (SR-IOV:
> false, MPS: true)
> 
> PMD: net_mlx5: 1 port(s) detected
> 
> PMD: net_mlx5: priv allocation failure
> 
> EAL: Requested device 0000:01:00.1 cannot be used
> 
> EAL: PCI device 0000:02:00.0 on NUMA socket 0
> 
> EAL:   probe driver: 8086:1521 net_e1000_igb
> 
> EAL: PCI device 0000:02:00.1 on NUMA socket 0
> 
> EAL:   probe driver: 8086:1521 net_e1000_igb
> 
> EAL: PCI device 0000:81:00.0 on NUMA socket -1
> 
> EAL:   probe driver: 15b3:1015 net_mlx5
> 
> PMD: net_mlx5: PCI information matches, using device "mlx5_2" (SR-IOV:
> false, MPS: true)
> 
> PMD: net_mlx5: 1 port(s) detected
> 
> PMD: net_mlx5: priv allocation failure
> 
> EAL: Requested device 0000:81:00.0 cannot be used
> 
> EAL: PCI device 0000:81:00.1 on NUMA socket -1
> 
> EAL:   probe driver: 15b3:1015 net_mlx5
> 
> PMD: net_mlx5: PCI information matches, using device "mlx5_3" (SR-IOV:
> false, MPS: true)
> 
> PMD: net_mlx5: 1 port(s) detected
> 
> PMD: net_mlx5: priv allocation failure
> 
> EAL: Requested device 0000:81:00.1 cannot be used
> 
> EAL: PCI device 0000:82:00.0 on NUMA socket -1
> 
> EAL:   probe driver: 8086:10fb net_ixgbe
> 
> EAL: PCI device 0000:82:00.1 on NUMA socket -1
> 
> EAL:   probe driver: 8086:10fb net_ixgbe
> 
> MAC updating enabled
> 
> MEMPOOL: Cannot allocate tailq entry!
> 
> EAL: Error - exiting with code: 1
> 
>   Cause: Cannot init mbuf pool
> 
> 
> *Hugepage info:*
> 
> $cat /sys/kernel/mm/hugepages/hugepages-2048kB/nr_hugepages
> 8192
> 
> $cat /sys/kernel/mm/hugepages/hugepages-2048kB/free_hugepages
> 5120
> 
> 
> But , If I run in the below command, it will run successful.
> 
> $*sudo* ./build/l2fwd -c 0x1 -n 4 -- -p 0x1
> 
> But, I dont want to run l2fwd in sudo mode.

A few extra capabilities are necessary in order to start a DPDK application
as an unprivileged user (as opposed to starting it as root before dropping
unwanted privileges).

Both mlx4 and mlx5 PMDs require CAP_NET_ADMIN, CAP_NET_RAW and CAP_IPC_LOCK
respectively to fiddle with their associated kernel netdevice, create queues
for any kind of traffic and lock memory (the last one is not necessarily
needed when running with --no-huge).

If acceptable for your application, then you may try something like:

 root# setcap cap_net_admin,cap_net_raw,cap_ipc_lock=ep ./build/l2fwd

 user$ ./build/l2fwd [...]

Note the described approach adds capabilities through extended device
attributes on the file system (those are rarely preserved when copying
files). While finer grained than "chmod +s", you still need to carefully
consider the security implications, trust the program and its users.

Have a look at man capabilities(7).

-- 
Adrien Mazarguil
6WIND