[dpdk-users] identifying physical dpdk-ports for use with OVS
Shivaram Mysore
shivaram.mysore at gmail.com
Thu Apr 6 18:18:21 CEST 2017
Hello,
I wanted to just share info about how we can identify DPDK ports for use
with OVS.
Initially, when we do:
# *dpdk-devbind --status*
Network devices using kernel driver
===================================
...
0000:82:00.0 'I350 Gigabit Network Connection' if=enp130s0f0 drv=igb
unused=vfio-pci
0000:82:00.1 'I350 Gigabit Network Connection' if=enp130s0f1 drv=igb
unused=vfio-pci
0000:82:00.2 'I350 Gigabit Network Connection' if=enp130s0f2 drv=igb
unused=vfio-pci
0000:82:00.3 'I350 Gigabit Network Connection' if=enp130s0f3 drv=igb
unused=vfio-pci
0000:83:00.0 'I350 Gigabit Network Connection' if=ens2f0 drv=igb
unused=vfio-pci
0000:83:00.1 'I350 Gigabit Network Connection' if=ens2f1 drv=igb
unused=vfio-pci
0000:83:00.2 'I350 Gigabit Network Connection' if=ens2f2 drv=igb
unused=vfio-pci
0000:83:00.3 'I350 Gigabit Network Connection' if=ens2f3 drv=igb
unused=vfio-pci
...
Then we do:
# *dpdk-devbind --bind=vfio-pci 0000:82:00.0 0000:82:00.1 0000:82:00.2
0000:82:00.3 0000:83:00.0 0000:83:00.1 0000:83:00.2 0000:83:00.3*
Now we do
# *dpdk-devbind --status*
Network devices using DPDK-compatible driver
============================================
0000:82:00.0 'I350 Gigabit Network Connection' drv=vfio-pci unused=igb
0000:82:00.1 'I350 Gigabit Network Connection' drv=vfio-pci unused=igb
0000:82:00.2 'I350 Gigabit Network Connection' drv=vfio-pci unused=igb
0000:82:00.3 'I350 Gigabit Network Connection' drv=vfio-pci unused=igb
0000:83:00.0 'I350 Gigabit Network Connection' drv=vfio-pci unused=igb
0000:83:00.1 'I350 Gigabit Network Connection' drv=vfio-pci unused=igb
0000:83:00.2 'I350 Gigabit Network Connection' drv=vfio-pci unused=igb
0000:83:00.3 'I350 Gigabit Network Connection' drv=vfio-pci unused=igb
--- oops we lost information about interface name (ex. ens2f0). This is
problem to identify physical ports if you don't have information about
PCI/slot address info and interface name mapping. Also, doing a ip a on
command line won't give you interface names as they are assigned to DPDK!!
So, to identify port by blinking led we cannot do something like:
# *ethtool -p enp130s0f0 5*
Cannot identify NIC: No such device
With the latest OVS (2.7), we can do (*note*: we no longer need "dpdk"
prefix to port name):
# ovs-vsctl add-port ovs-ip64-br0 enp130s0f0 -- set Interface enp130s0f0
type=dpdk options:dpdk-devargs=0000:82:00.0
# ovs-vsctl add-port ovs-ip64-br0 enp130s0f1 -- set Interface enp130s0f1
type=dpdk # options:dpdk-devargs=0000:82:00.1
# ovs-vsctl add-port ovs-ip64-br0 enp130s0f2 -- set Interface enp130s0f2
type=dpdk options:dpdk-devargs=0000:82:00.2
# ovs-vsctl add-port ovs-ip64-br0 enp130s0f3 -- set Interface enp130s0f3
type=dpdk options:dpdk-devargs=0000:82:00.3
# ovs-vsctl add-port ovs-ip64-br0 ens2f0 -- set Interface ens2f0 type=dpdk
options:dpdk-devargs=0000:83:00.0
# ovs-vsctl add-port ovs-ip64-br0 ens2f1 -- set Interface ens2f1 type=dpdk
options:dpdk-devargs=0000:83:00.1
# ovs-vsctl add-port ovs-ip64-br0 ens2f2 -- set Interface ens2f2 type=dpdk
# options:dpdk-devargs=0000:83:00.2
# ovs-vsctl add-port ovs-ip64-br0 ens2f3 -- set Interface ens2f3 type=dpdk
options:dpdk-devargs=0000:83:00.3
Which will result in:
# *ovs-vsctl show*
92e46601-ca42-4743-8d7d-838c8fff515e
Bridge "ovs-ip64-br0"
Controller "tcp:10.20.5.5:6653"
is_connected: true
Controller "tcp:10.20.5.5:6654"
is_connected: true
Port "ens2f0"
Interface "ens2f0"
type: dpdk
options: {dpdk-devargs="0000:83:00.0"}
Port "enp130s0f2"
Interface "enp130s0f2"
type: dpdk
options: {dpdk-devargs="0000:82:00.2"}
Port "ovs-ip64-br0"
Interface "ovs-ip64-br0"
type: internal
Port "ens2f1"
Interface "ens2f1"
type: dpdk
options: {dpdk-devargs="0000:83:00.1"}
Port "ens2f3"
Interface "ens2f3"
type: dpdk
options: {dpdk-devargs="0000:83:00.3"}
Port "enp130s0f0"
Interface "enp130s0f0"
type: dpdk
options: {dpdk-devargs="0000:82:00.0"}
Port "ens2f2"
Interface "ens2f2"
type: dpdk
options: {dpdk-devargs="0000:83:00.2"}
Port "enp130s0f1"
Interface "enp130s0f1"
type: dpdk
options: {dpdk-devargs="0000:82:00.1"}
Port "enp130s0f3"
Interface "enp130s0f3"
type: dpdk
options: {dpdk-devargs="0000:82:00.3"}
ovs_version: "2.7.0"
Now we know what ports to connect to.
Hope this helps
/Shivaram
More information about the users
mailing list