[dpdk-dev] performance issue with ovs + dpdk2.0 with vhost
Ravi Rao
Ravi.Rao at us.fujitsu.com
Fri May 15 18:11:43 CEST 2015
Hi,
This is what I am trying to Do.
Below is the setup..
| +----------------------+ |
| guest | |
| | |
| | | guest
| eth0 L3fwd eth1 | |
| | | | |
+---+--------------+---+ __|
^ :
| |
: v __
+-----------------+--------------+-----------------+ |
| | ovs-br0 | |
| +--------------+ | |
| ^ : | |
| +----------+ +---------+ | | host
| : v | |
| +--------------+ +--------------+ | |
| | dpdk0 | ovs-dpdk | dpdk1 | | |
+---+--------------+------------+--------------+---+ __|
^ :
| |
: v
+--------------------------------------------------+
| |
| traffic generator |
| |
+--------------------------------------------------+|
Step1: Use the latest ovs and dpdk2.0 to get the ovs running with 2 dpdk
interfaces that are bound to 2 10GB physical interfaces
#** Inser the required Modules
cd /root/dpdk-2.0.0
modprobe uio
modprobe cuse
rmmod igb_uio
rmmod rte_kni
insmod x86_64-native-linuxapp-gcc/kmod/igb_uio.ko
#**** Assign the dpdk capable interfaces to igb_uio driver
tools/dpdk_nic_bind.py --status
tools/dpdk_nic_bind.py -b igb_uio 0000:02:00.0
tools/dpdk_nic_bind.py -b igb_uio 0000:02:00.1
tools/dpdk_nic_bind.py --status
#--- Setup the openVswitch
cd /root/ovs
pkill -9 ovs
mkdir -p /usr/local/etc/openvswitch
mkdir -p /usr/local/var/run/openvswitch
rm -rf /usr/local/etc/openvswitch/conf.db
ovsdb/ovsdb-tool create /usr/local/etc/openvswitch/conf.db
vswitchd/vswitch.ovsschema
#Start ovsdb-server
ovsdb/ovsdb-server --remote=punix:/usr/local/var/run/openvswitch/db.sock
--remote=db:Open_vSwitch,Open_vSwitch,manager_options --pidfile --detach
utilities/ovs-vsctl --no-wait init
#Start vswitchd:
export DB_SOCK=/usr/local/var/run/openvswitch/db.sock
rm /dev/vhost-net
vswitchd/ovs-vswitchd --dpdk -c 0x3 -n 4 --socket-mem 1024,0 --
unix:$DB_SOCK --pidfile --detach
#Add bridge & ports
utilities/ovs-vsctl add-br ovs-br0 -- set bridge ovs-br0
datapath_type=netdev
utilities/ovs-vsctl add-port ovs-br0 dpdk0 -- set Interface dpdk0 type=dpdk
utilities/ovs-vsctl add-port ovs-br0 dpdk1 -- set Interface dpdk1 type=dpdk
Step2: Create the dpdkvhost interfaces and bring up the guestVM using QEMU
export DPDK_DIR=/root/dpdk-2.0.0
insmod $DPDK_DIR/lib/librte_vhost/eventfd_link/eventfd_link.ko
cd /root/ovs
utilities/ovs-vsctl add-port ovs-br0 dpdkvhost0 -- set Interface
dpdkvhost0 type=dpdkvhost
utilities/ovs-vsctl add-port ovs-br0 dpdkvhost1 -- set Interface
dpdkvhost1 type=dpdkvhost
#**** Start the guest ubuntu VM1 from a terminal that is logged in as root
qemu-system-x86_64 --enable-kvm -k fr -m 1G \
-cpu host -smp cores=2,threads=1,sockets=1 \
-serial telnet::4444,server,nowait -monitor
telnet::5555,server,nowait \
-hda /root/VMs/images/ubuntu-14.04-template.qcow2 \
-object
memory-backend-file,id=mem,size=1G,mem-path=/mnt/huge_1GB,share=on \
-numa node,memdev=mem \
-netdev
type=tap,id=dpdkvhost0,script=no,downscript=no,ifname=dpdkvhost0,vhost=on \
-device
virtio-net-pci,netdev=dpdkvhost0,mac=52:54:00:12:34:56,csum=off,gso=off,guest_tso4=off,guest_tso6=off,guest_ecn=off
\
-netdev
type=tap,id=dpdkvhost1,script=no,downscript=no,ifname=dpdkvhost1,vhost=on \
-device
virtio-net-pci,netdev=dpdkvhost1,mac=52:54:00:12:34:57,csum=off,gso=off,guest_tso4=off,guest_tso6=off,guest_ecn=off
\
-device ne2k_pci,mac=DE:AD:DE:01:02:03,netdev=user.0 -netdev
user,id=user.0,hostfwd=tcp::2222-:22 &
# **** Add flows between ports
utilities/ovs-ofctl del-flows ovs-br0
utilities/ovs-ofctl add-flow ovs-br0 in_port=1,action=output:3
utilities/ovs-ofctl add-flow ovs-br0 in_port=2,action=output:4
utilities/ovs-ofctl add-flow ovs-br0 in_port=3,action=output:1
utilities/ovs-ofctl add-flow ovs-br0 in_port=4,action=output:2
Once I complete the above settings. I log into the VM and enable ipv4
forwarding so that it could do l3 fwd between eth0 and eth1
The issue I am seeing is when I start pumping packets on IXIA port
connected to physical port dpdk0 I see lots of tx_errors on dpdk0.
I can only pass about 1000 pps without getting any errors. Is there
anything I am doing wrong or missing in the above setup.
Thanks & Regards,
Ravi..
On 05/15/15 09:41, Thomas F Herbert wrote:
> On 5/15/15 9:55 AM, Rao, Ravi wrote:
>> Hi All,
>> I am trying to get a Guest VM connected to the dpdkvhost
>> interface on a host which has the ovs running from the latest ovs git
>> and dpdk2.0. Looks like I am missing something as 95% of traffic is
>> not getting to the VM. Can one of you please let me know which
>> mailing list I should be posting the details for getting a
>> resolution. Is it this dpdk list OR would it be the openvswitch list?
> You could also try the openvswitch discuss list but this list is a
> good starting point. I am working with vhost/dpdk2.0 and ovs and
> others are too.
> Could you please send more details about what you are seeing? Are you
> using the Linux stack in the guest? Do you see packets on the input IF
> in the guest?
>> Regards,
>> Ravi..
>
>
More information about the dev
mailing list