[dpdk-dev] Need help with the correct steps to run testpmd app on Linux VM using VMXNET3 vNICs

Sandesh Sawant ssawant at vmware.com
Fri Nov 21 14:35:46 CET 2014


Hello dpdk experts,

I am following the documentation available on dpdk.org & the documentation published by Intel to learn how to use DPDK libraries.

I am using dpdk v1.7.1 downloaded from dpdk.org. I wish to run testpmd app provided with this release with 2 VMXNET3 vNICs on my setup.
My test setup is a 64-bit Ubunutu 14.04 (3.13.0) Linux VM running on a VMWare ESXi-5.5.0 hypervisor on Dell server with Intel Xeon E5-2440.

I followed the following stpes to capture the vmxnet3 vNICs in DPDK (all commands run as a root user):

cd /home/sandesh/dpdk/dpdk-1.7.1/
export RTE_SDK=/home/sandesh/dpdk/dpdk-1.7.1/
export RTE_TARGET=build
modprobe uio
insmod build/kmod/igb_uio.ko
modprobe vfio-pci
mkdir -p /mnt/huge
mount -t hugetlbfs nodev /mnt/huge
echo 64 > /sys/devices/system/node/node0/hugepages/hugepages-2048kB/nr_hugepages

tools/dpdk_nic_bind.py --status

Network devices using DPDK-compatible driver
============================================
<none>

Network devices using kernel driver
===================================
0000:02:01.0 '82545EM Gigabit Ethernet Controller (Copper)' if=eth2 drv=e1000 unused=igb_uio,vfio-pci *Active*
0000:03:00.0 'VMXNET3 Ethernet Controller' if=eth0 drv=vmxnet3 unused=igb_uio,vfio-pci *Active*
0000:0b:00.0 'VMXNET3 Ethernet Controller' if=eth1 drv=vmxnet3 unused=igb_uio,vfio-pci *Active*

Other network devices
=====================
<none>

Then I bind VMXNET3 vNICs to igb_uio driver:

../dpdk-1.7.1/tools/dpdk_nic_bind.py --bind=igb_uio eth0 eth1

../dpdk-1.7.1/tools/dpdk_nic_bind.py --status

Network devices using DPDK-compatible driver
============================================
<none>

Network devices using kernel driver
===================================
0000:02:01.0 '82545EM Gigabit Ethernet Controller (Copper)' if=eth2 drv=e1000 unused=igb_uio,vfio-pci *Active*
0000:03:00.0 'VMXNET3 Ethernet Controller' if=eth0 drv=vmxnet3 unused=igb_uio,vfio-pci *Active*
0000:0b:00.0 'VMXNET3 Ethernet Controller' if=eth1 drv=vmxnet3 unused=igb_uio,vfio-pci *Active*

Other network devices
=====================
<none>


Note: I downloaded vmxnet3-usermap extension 1.2 source from dpdk.org to work with VMXNET3.
I got a few compilation errors for the same which I resolved and got the driver & .so built.
(I belive these errors in driver code were due to its code not being uptodate wrt linux kernel 3.13).

Next I proceeded to replace the orignial vmxnet3 driver with vmxnet3-usermap.ko as follows:

cd /home/sandesh/dpdk/vmxnet3-usermap-1.2/
rmmod vmxnet3
insmod vmxnet3-usermap.ko enable_shm=2,2 num_rqs=1,1 num_rxds=512 num_txds=512

and then executed:

../dpdk-1.7.1/build/app/testpmd -c 0xff -n 1 -d librte_pmd_vmxnet3.so -- -i --rxq=1 --rxd=512 --txd=512

EAL: Detected lcore 0 as core 0 on socket 0
EAL: Detected lcore 1 as core 1 on socket 0
EAL: Detected lcore 2 as core 2 on socket 0
EAL: Detected lcore 3 as core 3 on socket 0
EAL: Detected lcore 4 as core 0 on socket 0
EAL: Detected lcore 5 as core 1 on socket 0
EAL: Detected lcore 6 as core 2 on socket 0
EAL: Detected lcore 7 as core 3 on socket 0
EAL: Support maximum 64 logical core(s) by configuration.
EAL: Detected 8 lcore(s)
EAL:   unsupported IOMMU type!
EAL: VFIO support could not be initialized
EAL: Setting up memory...
EAL: Ask a virtual area of 0x5800000 bytes
EAL: Virtual area found at 0x7fb8f7000000 (size = 0x5800000)
EAL: Ask a virtual area of 0x400000 bytes
EAL: Virtual area found at 0x7fb8f6a00000 (size = 0x400000)
EAL: Ask a virtual area of 0x2400000 bytes
EAL: Virtual area found at 0x7fb8f4400000 (size = 0x2400000)
EAL: Requesting 64 pages of size 2MB from socket 0
EAL: TSC frequency is ~2400001 KHz
EAL: open shared lib /home/sandesh/dpdk/vmxnet3-usermap-1.2/librte_pmd_vmxnet3.so
librte_pmd_vmxnet3 version 1.2
        Copyright 2012-2014 6WIND S.A. provided without warranty.
EAL: Master core 0 is ready (tid=fd716880)
EAL: Core 7 is ready (tid=f09ed700)
EAL: Core 6 is ready (tid=f11ee700)
EAL: Core 5 is ready (tid=f19ef700)
EAL: Core 4 is ready (tid=f21f0700)
EAL: Core 3 is ready (tid=f29f1700)
EAL: Core 2 is ready (tid=f31f2700)
EAL: Core 1 is ready (tid=f39f3700)
EAL: PCI device 0000:02:01.0 on NUMA socket -1
EAL:   probe driver: 8086:100f rte_em_pmd
EAL:   0000:02:01.0 not managed by UIO driver, skipping
EAL: PCI device 0000:03:00.0 on NUMA socket -1
EAL:   probe driver: 15ad:7b0 rte_vmxnet3_pmd
PMD: eth_vmxnet3_dev_init(): No matching device found
EAL: Error - exiting with code: 1
  Cause: Requested device 0000:03:00.0 cannot be used
root at pnq1-vm-eng-dhcp13:/home/sandesh/dpdk/vmxnet3-usermap-1.2#


Can someone please point out what I am missing here?
Am I supposed to bind VMXNET3 vNICs to the vmxnet3-usermap driver instead of igb_uio driver here?


Note: Earlier when I was not running testpmd without vmxnet3-usermap extension, when I was hitting the following error:
dpdk-1.7.1/build/app/testpmd -c 0xff -n 1 -- -i
.
.
.
EAL: PCI device 0000:0b:00.0 on NUMA socket -1
EAL:   probe driver: 15ad:7b0 rte_vmxnet3_pmd
EAL:   PCI memory mapped at 0x7f945efe7000
EAL:   PCI memory mapped at 0x7f945efe6000
EAL:   PCI memory mapped at 0x7f945efe4000
Interactive-mode selected
Configuring Port 0 (socket 0)
Fail to configure port 0 tx queues
EAL: Error - exiting with code: 1
  Cause: Start ports failed

Please let me know the correct method to run this application.
Thanks in advance.

Best,
Sandesh



More information about the dev mailing list