[dpdk-users] mellanox connect-x 3 pro dpdk with openvswitch

Raslan Darawsheh rasland at mellanox.com
Mon Jun 11 13:19:57 CEST 2018


>From what I'm seeing it looks like it didn't prop the MLX4 device at all. 
Can you make sure you have it enabled in your DPDK package when you compiled DPDK ?

Kindest regards,
Raslan Darawsheh

-----Original Message-----
From: Tobias Urdin [mailto:tobias.urdin at crystone.com] 
Sent: Monday, June 11, 2018 2:16 PM
To: Raslan Darawsheh <rasland at mellanox.com>
Cc: users at dpdk.org
Subject: Re: mellanox connect-x 3 pro dpdk with openvswitch

Ok, here is the complete log when restarting the openvswitch-switch service, see below.

Also here is the kernel for information:
$uname -r
4.4.0-127-generic

2018-06-11T11:13:42.729Z|00176|bridge|INFO|bridge br-mlx: deleted interface br-mlx on port 65534 2018-06-11T11:13:43.274Z|00177|bridge|INFO|bridge br-int: deleted interface br-int on port 65534 2018-06-11T11:13:43.275Z|00178|bridge|INFO|bridge br-int: deleted interface int-br-ex on port 1 2018-06-11T11:13:43.275Z|00179|bridge|INFO|bridge br-int: deleted interface patch-tun on port 2 2018-06-11T11:13:43.762Z|00180|bridge|INFO|bridge br-tun: deleted interface patch-int on port 1 2018-06-11T11:13:43.762Z|00181|bridge|INFO|bridge br-tun: deleted interface br-tun on port 65534 2018-06-11T11:13:44.231Z|00182|bridge|INFO|bridge br-ex: deleted interface br-ex on port 65534 2018-06-11T11:13:44.232Z|00183|bridge|INFO|bridge br-ex: deleted interface phy-br-ex on port 1 2018-06-11T11:13:45.245Z|00001|vlog|INFO|opened log file /var/log/openvswitch/ovs-vswitchd.log
2018-06-11T11:13:45.252Z|00002|ovs_numa|INFO|Discovered 40 CPU cores on NUMA node 0 2018-06-11T11:13:45.252Z|00003|ovs_numa|INFO|Discovered 40 CPU cores on NUMA node 1 2018-06-11T11:13:45.252Z|00004|ovs_numa|INFO|Discovered 2 NUMA nodes and
80 CPU cores
2018-06-11T11:13:45.252Z|00005|reconnect|INFO|unix:/var/run/openvswitch/db.sock:
connecting...
2018-06-11T11:13:45.252Z|00006|reconnect|INFO|unix:/var/run/openvswitch/db.sock:
connected
2018-06-11T11:13:45.256Z|00007|dpdk|INFO|Using DPDK 17.11.0 2018-06-11T11:13:45.256Z|00008|dpdk|INFO|DPDK Enabled - initializing...
2018-06-11T11:13:45.256Z|00009|dpdk|INFO|No vhost-sock-dir provided - defaulting to /var/run/openvswitch 2018-06-11T11:13:45.256Z|00010|dpdk|INFO|IOMMU support for vhost-user-client disabled.
2018-06-11T11:13:45.256Z|00011|dpdk|INFO|EAL ARGS: ovs-vswitchd --socket-mem 1024,0 -c 0x00000001
2018-06-11T11:13:45.258Z|00012|dpdk|INFO|EAL: Detected 80 lcore(s)
2018-06-11T11:13:45.285Z|00013|dpdk|INFO|EAL: Probing VFIO support...
2018-06-11T11:13:45.285Z|00014|dpdk|INFO|EAL: VFIO support initialized
2018-06-11T11:13:48.214Z|00015|dpdk|INFO|EAL: PCI device 0000:01:00.0 on NUMA socket 0
2018-06-11T11:13:48.214Z|00016|dpdk|INFO|EAL:   probe driver: 8086:1521
net_e1000_igb
2018-06-11T11:13:48.214Z|00017|dpdk|INFO|EAL: PCI device 0000:01:00.1 on NUMA socket 0
2018-06-11T11:13:48.214Z|00018|dpdk|INFO|EAL:   probe driver: 8086:1521
net_e1000_igb
2018-06-11T11:13:48.214Z|00019|dpdk|INFO|EAL: PCI device 0000:18:00.0 on NUMA socket 0
2018-06-11T11:13:48.214Z|00020|dpdk|INFO|EAL:   probe driver: 8086:1572
net_i40e
2018-06-11T11:13:48.214Z|00021|dpdk|INFO|EAL: PCI device 0000:18:00.1 on NUMA socket 0
2018-06-11T11:13:48.214Z|00022|dpdk|INFO|EAL:   probe driver: 8086:1572
net_i40e
2018-06-11T11:13:48.214Z|00023|dpdk|INFO|DPDK pdump packet capture enabled 2018-06-11T11:13:48.224Z|00024|dpdk|INFO|DPDK Enabled - initialized
2018-06-11T11:13:48.235Z|00025|ofproto_dpif|INFO|netdev at ovs-netdev:
Datapath supports recirculation
2018-06-11T11:13:48.235Z|00026|ofproto_dpif|INFO|netdev at ovs-netdev: VLAN header stack length probed as 1
2018-06-11T11:13:48.235Z|00027|ofproto_dpif|INFO|netdev at ovs-netdev: MPLS label stack length probed as 3
2018-06-11T11:13:48.235Z|00028|ofproto_dpif|INFO|netdev at ovs-netdev:
Datapath supports truncate action
2018-06-11T11:13:48.235Z|00029|ofproto_dpif|INFO|netdev at ovs-netdev:
Datapath supports unique flow ids
2018-06-11T11:13:48.235Z|00030|ofproto_dpif|INFO|netdev at ovs-netdev:
Datapath supports clone action
2018-06-11T11:13:48.235Z|00031|ofproto_dpif|INFO|netdev at ovs-netdev: Max sample nesting level probed as 10
2018-06-11T11:13:48.235Z|00032|ofproto_dpif|INFO|netdev at ovs-netdev:
Datapath supports eventmask in conntrack action
2018-06-11T11:13:48.235Z|00033|ofproto_dpif|INFO|netdev at ovs-netdev:
Datapath supports ct_clear action
2018-06-11T11:13:48.235Z|00034|ofproto_dpif|INFO|netdev at ovs-netdev:
Datapath supports ct_state
2018-06-11T11:13:48.235Z|00035|ofproto_dpif|INFO|netdev at ovs-netdev:
Datapath supports ct_zone
2018-06-11T11:13:48.235Z|00036|ofproto_dpif|INFO|netdev at ovs-netdev:
Datapath supports ct_mark
2018-06-11T11:13:48.235Z|00037|ofproto_dpif|INFO|netdev at ovs-netdev:
Datapath supports ct_label
2018-06-11T11:13:48.235Z|00038|ofproto_dpif|INFO|netdev at ovs-netdev:
Datapath supports ct_state_nat
2018-06-11T11:13:48.235Z|00039|ofproto_dpif|INFO|netdev at ovs-netdev:
Datapath supports ct_orig_tuple
2018-06-11T11:13:48.235Z|00040|ofproto_dpif|INFO|netdev at ovs-netdev:
Datapath supports ct_orig_tuple6
2018-06-11T11:13:48.605Z|00041|ofproto_dpif|INFO|system at ovs-system:
Datapath supports recirculation
2018-06-11T11:13:48.605Z|00042|ofproto_dpif|INFO|system at ovs-system: VLAN header stack length probed as 1
2018-06-11T11:13:48.605Z|00043|ofproto_dpif|INFO|system at ovs-system: MPLS label stack length probed as 1
2018-06-11T11:13:48.605Z|00044|ofproto_dpif|INFO|system at ovs-system:
Datapath does not support truncate action
2018-06-11T11:13:48.605Z|00045|ofproto_dpif|INFO|system at ovs-system:
Datapath supports unique flow ids
2018-06-11T11:13:48.605Z|00046|ofproto_dpif|INFO|system at ovs-system:
Datapath does not support clone action
2018-06-11T11:13:48.605Z|00047|ofproto_dpif|INFO|system at ovs-system: Max sample nesting level probed as 3
2018-06-11T11:13:48.605Z|00048|ofproto_dpif|INFO|system at ovs-system:
Datapath does not support eventmask in conntrack action
2018-06-11T11:13:48.605Z|00049|ofproto_dpif|INFO|system at ovs-system:
Datapath does not support ct_clear action
2018-06-11T11:13:48.605Z|00050|ofproto_dpif|INFO|system at ovs-system:
Datapath supports ct_state
2018-06-11T11:13:48.605Z|00051|ofproto_dpif|INFO|system at ovs-system:
Datapath supports ct_zone
2018-06-11T11:13:48.605Z|00052|ofproto_dpif|INFO|system at ovs-system:
Datapath supports ct_mark
2018-06-11T11:13:48.605Z|00053|ofproto_dpif|INFO|system at ovs-system:
Datapath supports ct_label
2018-06-11T11:13:48.605Z|00054|ofproto_dpif|INFO|system at ovs-system:
Datapath does not support ct_state_nat
2018-06-11T11:13:48.605Z|00055|ofproto_dpif|INFO|system at ovs-system:
Datapath does not support ct_orig_tuple
2018-06-11T11:13:48.605Z|00056|ofproto_dpif|INFO|system at ovs-system:
Datapath does not support ct_orig_tuple6 2018-06-11T11:13:48.981Z|00057|bridge|INFO|bridge br-mlx: added interface br-mlx on port 65534
2018-06-11T11:13:48.981Z|00058|dpdk|ERR|EAL: Unable to find a bus for the device 'mlx4_0'
2018-06-11T11:13:48.981Z|00059|netdev_dpdk|WARN|Error attaching device 'mlx4_0' to DPDK
2018-06-11T11:13:48.981Z|00060|netdev|WARN|enp59s0d1: could not set configuration (Invalid argument) 2018-06-11T11:13:48.982Z|00061|bridge|INFO|bridge br-int: added interface br-int on port 65534 2018-06-11T11:13:48.982Z|00062|bridge|INFO|bridge br-int: added interface int-br-ex on port 1 2018-06-11T11:13:48.982Z|00063|bridge|INFO|bridge br-int: added interface patch-tun on port 2 2018-06-11T11:13:48.982Z|00064|bridge|INFO|bridge br-tun: added interface patch-int on port 1 2018-06-11T11:13:48.982Z|00065|bridge|INFO|bridge br-tun: added interface br-tun on port 65534 2018-06-11T11:13:48.982Z|00066|bridge|INFO|bridge br-ex: added interface br-ex on port 65534 2018-06-11T11:13:48.982Z|00067|bridge|INFO|bridge br-ex: added interface phy-br-ex on port 1 2018-06-11T11:13:48.982Z|00068|bridge|INFO|bridge br-mlx: using datapath ID 000072528484a642
2018-06-11T11:13:48.982Z|00069|connmgr|INFO|br-mlx: added service controller "punix:/var/run/openvswitch/br-mlx.mgmt"
2018-06-11T11:13:49.221Z|00070|bridge|INFO|bridge br-int: using datapath ID 00006680927b6b40
2018-06-11T11:13:49.221Z|00071|connmgr|INFO|br-int: added service controller "punix:/var/run/openvswitch/br-int.mgmt"
2018-06-11T11:13:49.221Z|00072|connmgr|INFO|br-int: added primary controller "tcp:127.0.0.1:6633"
2018-06-11T11:13:49.221Z|00073|rconn|INFO|br-int<->tcp:127.0.0.1:6633:
connecting...
2018-06-11T11:13:49.460Z|00074|bridge|INFO|bridge br-tun: using datapath ID 000066532e5f0443
2018-06-11T11:13:49.461Z|00075|connmgr|INFO|br-tun: added service controller "punix:/var/run/openvswitch/br-tun.mgmt"
2018-06-11T11:13:49.461Z|00076|connmgr|INFO|br-tun: added primary controller "tcp:127.0.0.1:6633"
2018-06-11T11:13:49.461Z|00077|rconn|INFO|br-tun<->tcp:127.0.0.1:6633:
connecting...
2018-06-11T11:13:49.692Z|00078|bridge|INFO|bridge br-ex: using datapath ID 0000e2d1bdbe8e45
2018-06-11T11:13:49.692Z|00079|connmgr|INFO|br-ex: added service controller "punix:/var/run/openvswitch/br-ex.mgmt"
2018-06-11T11:13:49.692Z|00080|connmgr|INFO|br-ex: added primary controller "tcp:127.0.0.1:6633"
2018-06-11T11:13:49.692Z|00081|rconn|INFO|br-ex<->tcp:127.0.0.1:6633:
connecting...
2018-06-11T11:13:49.916Z|00082|timeval|WARN|Unreasonably long 4661ms poll interval (16ms user, 4472ms system)
2018-06-11T11:13:49.916Z|00083|timeval|WARN|faults: 3584706 minor, 0 major
2018-06-11T11:13:49.916Z|00084|timeval|WARN|disk: 0 reads, 16 writes 2018-06-11T11:13:49.916Z|00085|timeval|WARN|context switches: 147 voluntary, 25 involuntary 2018-06-11T11:13:49.916Z|00086|coverage|INFO|Event coverage, avg rate over last: 5 seconds, last minute, last hour,  hash=2ba35161:
2018-06-11T11:13:49.916Z|00087|coverage|INFO|bridge_reconfigure        
0.0/sec     0.000/sec        0.0000/sec   total: 1
2018-06-11T11:13:49.916Z|00088|coverage|INFO|ofproto_flush             
0.0/sec     0.000/sec        0.0000/sec   total: 4
2018-06-11T11:13:49.916Z|00089|coverage|INFO|ofproto_update_port       
0.0/sec     0.000/sec        0.0000/sec   total: 8
2018-06-11T11:13:49.916Z|00090|coverage|INFO|rev_flow_table            
0.0/sec     0.000/sec        0.0000/sec   total: 1
2018-06-11T11:13:49.916Z|00091|coverage|INFO|cmap_expand               
0.0/sec     0.000/sec        0.0000/sec   total: 52
2018-06-11T11:13:49.916Z|00092|coverage|INFO|cmap_shrink               
0.0/sec     0.000/sec        0.0000/sec   total: 23
2018-06-11T11:13:49.916Z|00093|coverage|INFO|dpif_port_add             
0.0/sec     0.000/sec        0.0000/sec   total: 1
2018-06-11T11:13:49.916Z|00094|coverage|INFO|dpif_flow_flush           
0.0/sec     0.000/sec        0.0000/sec   total: 6
2018-06-11T11:13:49.916Z|00095|coverage|INFO|dpif_flow_get             
0.0/sec     0.000/sec        0.0000/sec   total: 29
2018-06-11T11:13:49.916Z|00096|coverage|INFO|dpif_flow_put             
0.0/sec     0.000/sec        0.0000/sec   total: 37
2018-06-11T11:13:49.916Z|00097|coverage|INFO|dpif_flow_del             
0.0/sec     0.000/sec        0.0000/sec   total: 29
2018-06-11T11:13:49.916Z|00098|coverage|INFO|dpif_execute              
0.0/sec     0.000/sec        0.0000/sec   total: 8
2018-06-11T11:13:49.916Z|00099|coverage|INFO|flow_extract              
0.0/sec     0.000/sec        0.0000/sec   total: 6
2018-06-11T11:13:49.916Z|00100|coverage|INFO|miniflow_malloc           
0.0/sec     0.000/sec        0.0000/sec   total: 34
2018-06-11T11:13:49.916Z|00101|coverage|INFO|hmap_expand               
0.0/sec     0.000/sec        0.0000/sec   total: 498
2018-06-11T11:13:49.916Z|00102|coverage|INFO|netdev_received           
0.0/sec     0.000/sec        0.0000/sec   total: 1
2018-06-11T11:13:49.916Z|00103|coverage|INFO|netdev_get_stats          
0.0/sec     0.000/sec        0.0000/sec   total: 8
2018-06-11T11:13:49.916Z|00104|coverage|INFO|poll_create_node          
0.0/sec     0.000/sec        0.0000/sec   total: 23
2018-06-11T11:13:49.916Z|00105|coverage|INFO|seq_change                
0.0/sec     0.000/sec        0.0000/sec   total: 2193
2018-06-11T11:13:49.916Z|00106|coverage|INFO|pstream_open              
0.0/sec     0.000/sec        0.0000/sec   total: 9
2018-06-11T11:13:49.916Z|00107|coverage|INFO|stream_open               
0.0/sec     0.000/sec        0.0000/sec   total: 4
2018-06-11T11:13:49.916Z|00108|coverage|INFO|util_xalloc               
0.0/sec     0.000/sec        0.0000/sec   total: 12803
2018-06-11T11:13:49.916Z|00109|coverage|INFO|vconn_open                
0.0/sec     0.000/sec        0.0000/sec   total: 3
2018-06-11T11:13:49.916Z|00110|coverage|INFO|netdev_set_policing       
0.0/sec     0.000/sec        0.0000/sec   total: 4
2018-06-11T11:13:49.916Z|00111|coverage|INFO|netdev_get_ifindex        
0.0/sec     0.000/sec        0.0000/sec   total: 4
2018-06-11T11:13:49.916Z|00112|coverage|INFO|netdev_get_hwaddr         
0.0/sec     0.000/sec        0.0000/sec   total: 9
2018-06-11T11:13:49.916Z|00113|coverage|INFO|netdev_get_ethtool        
0.0/sec     0.000/sec        0.0000/sec   total: 2
2018-06-11T11:13:49.916Z|00114|coverage|INFO|netlink_received          
0.0/sec     0.000/sec        0.0000/sec   total: 85
2018-06-11T11:13:49.916Z|00115|coverage|INFO|netlink_recv_jumbo        
0.0/sec     0.000/sec        0.0000/sec   total: 1
2018-06-11T11:13:49.916Z|00116|coverage|INFO|netlink_sent              
0.0/sec     0.000/sec        0.0000/sec   total: 102
2018-06-11T11:13:49.916Z|00117|coverage|INFO|74 events never hit 2018-06-11T11:13:49.916Z|00118|poll_loop|INFO|wakeup due to [POLLIN] on fd 433 (FIFO pipe:[1025715]) at ../lib/ovs-rcu.c:229 (97% CPU usage) 2018-06-11T11:13:49.917Z|00119|poll_loop|INFO|wakeup due to [POLLIN] on fd 433 (FIFO pipe:[1025715]) at ../lib/ovs-rcu.c:229 (97% CPU usage) 2018-06-11T11:13:49.917Z|00120|poll_loop|INFO|wakeup due to 0-ms timeout at ../lib/ovs-rcu.c:229 (97% CPU usage) 2018-06-11T11:13:49.917Z|00121|poll_loop|INFO|wakeup due to 0-ms timeout at ../lib/ovs-rcu.c:229 (97% CPU usage)
2018-06-11T11:13:49.920Z|00122|rconn|INFO|br-int<->tcp:127.0.0.1:6633:
connected
2018-06-11T11:13:49.920Z|00123|rconn|INFO|br-tun<->tcp:127.0.0.1:6633:
connected
2018-06-11T11:13:49.920Z|00124|rconn|INFO|br-ex<->tcp:127.0.0.1:6633:
connected
2018-06-11T11:13:49.926Z|00125|poll_loop|INFO|wakeup due to [POLLIN] on fd 10 (NETLINK_ROUTE<->NETLINK_ROUTE) at ../lib/netlink-socket.c:1331 (97% CPU usage) 2018-06-11T11:13:49.926Z|00126|poll_loop|INFO|wakeup due to [POLLIN] on fd 421 (character device /dev/net/tun) at ../lib/netdev-linux.c:1171 (97% CPU usage) 2018-06-11T11:13:49.926Z|00127|poll_loop|INFO|wakeup due to [POLLIN] on fd 423 (127.0.0.1:33712<->127.0.0.1:6633) at ../lib/stream-fd.c:157 (97% CPU usage) 2018-06-11T11:13:49.926Z|00128|poll_loop|INFO|wakeup due to [POLLIN] on fd 428 (127.0.0.1:33714<->127.0.0.1:6633) at ../lib/stream-fd.c:157 (97% CPU usage) 2018-06-11T11:13:49.926Z|00129|poll_loop|INFO|wakeup due to [POLLIN] on fd 11 (<->/var/run/openvswitch/db.sock) at ../lib/stream-fd.c:157 (97% CPU usage) 2018-06-11T11:13:49.926Z|00130|poll_loop|INFO|wakeup due to [POLLIN] on fd 429 (127.0.0.1:33716<->127.0.0.1:6633) at ../lib/stream-fd.c:157 (97% CPU usage)
2018-06-11T11:13:49.930Z|00131|dpdk|ERR|EAL: Unable to find a bus for the device 'mlx4_0'
2018-06-11T11:13:49.930Z|00132|netdev_dpdk|WARN|Error attaching device 'mlx4_0' to DPDK
2018-06-11T11:13:49.930Z|00133|netdev|WARN|enp59s0d1: could not set configuration (Invalid argument) 2018-06-11T11:13:49.931Z|00134|bridge|INFO|ovs-vswitchd (Open vSwitch) 2.9.0
2018-06-11T11:13:53.662Z|00135|dpdk|ERR|EAL: Unable to find a bus for the device 'mlx4_0'
2018-06-11T11:13:53.662Z|00136|netdev_dpdk|WARN|Error attaching device 'mlx4_0' to DPDK
2018-06-11T11:13:53.662Z|00137|netdev|WARN|enp59s0d1: could not set configuration (Invalid argument)
2018-06-11T11:13:53.675Z|00138|dpdk|ERR|EAL: Unable to find a bus for the device 'mlx4_0'
2018-06-11T11:13:53.675Z|00139|netdev_dpdk|WARN|Error attaching device 'mlx4_0' to DPDK
2018-06-11T11:13:53.675Z|00140|netdev|WARN|enp59s0d1: could not set configuration (Invalid argument)
2018-06-11T11:13:53.682Z|00141|dpdk|ERR|EAL: Unable to find a bus for the device 'mlx4_0'
2018-06-11T11:13:53.682Z|00142|netdev_dpdk|WARN|Error attaching device 'mlx4_0' to DPDK
2018-06-11T11:13:53.682Z|00143|netdev|WARN|enp59s0d1: could not set configuration (Invalid argument)
2018-06-11T11:13:53.693Z|00144|dpdk|ERR|EAL: Unable to find a bus for the device 'mlx4_0'
2018-06-11T11:13:53.693Z|00145|netdev_dpdk|WARN|Error attaching device 'mlx4_0' to DPDK
2018-06-11T11:13:53.693Z|00146|netdev|WARN|enp59s0d1: could not set configuration (Invalid argument)
2018-06-11T11:13:53.705Z|00147|dpdk|ERR|EAL: Unable to find a bus for the device 'mlx4_0'
2018-06-11T11:13:53.705Z|00148|netdev_dpdk|WARN|Error attaching device 'mlx4_0' to DPDK
2018-06-11T11:13:53.705Z|00149|netdev|WARN|enp59s0d1: could not set configuration (Invalid argument)
2018-06-11T11:13:53.711Z|00150|dpdk|ERR|EAL: Unable to find a bus for the device 'mlx4_0'
2018-06-11T11:13:53.711Z|00151|netdev_dpdk|WARN|Error attaching device 'mlx4_0' to DPDK
2018-06-11T11:13:53.711Z|00152|netdev|WARN|enp59s0d1: could not set configuration (Invalid argument)
2018-06-11T11:13:53.717Z|00153|dpdk|ERR|EAL: Unable to find a bus for the device 'mlx4_0'
2018-06-11T11:13:53.717Z|00154|netdev_dpdk|WARN|Error attaching device 'mlx4_0' to DPDK
2018-06-11T11:13:53.717Z|00155|netdev|WARN|enp59s0d1: could not set configuration (Invalid argument)
2018-06-11T11:13:53.728Z|00156|dpdk|ERR|EAL: Unable to find a bus for the device 'mlx4_0'
2018-06-11T11:13:53.728Z|00157|netdev_dpdk|WARN|Error attaching device 'mlx4_0' to DPDK
2018-06-11T11:13:53.728Z|00158|netdev|WARN|enp59s0d1: could not set configuration (Invalid argument)
2018-06-11T11:13:53.743Z|00159|dpdk|ERR|EAL: Unable to find a bus for the device 'mlx4_0'
2018-06-11T11:13:53.743Z|00160|netdev_dpdk|WARN|Error attaching device 'mlx4_0' to DPDK
2018-06-11T11:13:53.743Z|00161|netdev|WARN|enp59s0d1: could not set configuration (Invalid argument)
2018-06-11T11:13:53.749Z|00162|dpdk|ERR|EAL: Unable to find a bus for the device 'mlx4_0'
2018-06-11T11:13:53.749Z|00163|netdev_dpdk|WARN|Error attaching device 'mlx4_0' to DPDK
2018-06-11T11:13:53.749Z|00164|netdev|WARN|enp59s0d1: could not set configuration (Invalid argument)
2018-06-11T11:13:53.756Z|00165|dpdk|ERR|EAL: Unable to find a bus for the device 'mlx4_0'
2018-06-11T11:13:53.756Z|00166|netdev_dpdk|WARN|Error attaching device 'mlx4_0' to DPDK
2018-06-11T11:13:53.756Z|00167|netdev|WARN|enp59s0d1: could not set configuration (Invalid argument)
2018-06-11T11:13:53.766Z|00168|dpdk|ERR|EAL: Unable to find a bus for the device 'mlx4_0'
2018-06-11T11:13:53.766Z|00169|netdev_dpdk|WARN|Error attaching device 'mlx4_0' to DPDK
2018-06-11T11:13:53.766Z|00170|netdev|WARN|enp59s0d1: could not set configuration (Invalid argument)
2018-06-11T11:13:55.270Z|00171|memory|INFO|1406100 kB peak resident set size after 10.0 seconds
2018-06-11T11:13:55.270Z|00172|memory|INFO|handlers:118 ofconns:3
ports:8 revalidators:42 rules:36 udpif keys:1
2018-06-11T11:14:03.699Z|00173|connmgr|INFO|br-int<->tcp:127.0.0.1:6633:
5 flow_mods 10 s ago (5 adds)
2018-06-11T11:14:03.737Z|00174|connmgr|INFO|br-ex<->tcp:127.0.0.1:6633:
2 flow_mods 10 s ago (2 adds)
2018-06-11T11:14:03.773Z|00175|connmgr|INFO|br-tun<->tcp:127.0.0.1:6633:
12 flow_mods 10 s ago (12 adds)

Best regards

On 06/11/2018 12:57 PM, Raslan Darawsheh wrote:
> What I really want to see is the init process for OVS-DPDK that dumps the DPDK initialization.
> I think you will find it at the beginning of the log that you just sent.
>
> Kindest regards,
> Raslan Darawsheh
>
> -----Original Message-----
> From: Tobias Urdin [mailto:tobias.urdin at crystone.com]
> Sent: Monday, June 11, 2018 1:00 PM
> To: Raslan Darawsheh <rasland at mellanox.com>
> Cc: users at dpdk.org
> Subject: Re: mellanox connect-x 3 pro dpdk with openvswitch
>
> Hello Raslan,
> Thank you for your reply.
>
> For reference the br-mlx bridge was created like this:
> $ovs-vsctl add-br br-mlx -- set bridge br-mlx datapath_type=netdev
>
> $ovs-vsctl add-port br-mlx enp59s0d1 -- set Interface enp59s0d1 
> type=dpdk options:dpdk-devargs=mlx4_0
> ovs-vsctl: Error detected while setting up 'enp59s0d1': Error attaching device 'mlx4_0' to DPDK.  See ovs-vswitchd log for details.
> ovs-vsctl: The default log directory is "/var/log/openvswitch".
>
> In the ovs-vswitchd.log
> 2018-06-11T09:56:20.613Z|00185|dpdk|ERR|EAL: Unable to find a bus for the device 'mlx4_0'
> 2018-06-11T09:56:20.613Z|00186|netdev_dpdk|WARN|Error attaching device 
> 'mlx4_0' to DPDK
> 2018-06-11T09:56:20.613Z|00187|netdev|WARN|enp59s0d1: could not set 
> configuration (Invalid argument)
>
> Here is the complete log after restarting openvswitch-switch service:
> 2018-06-11T09:57:49.380Z|00134|bridge|INFO|ovs-vswitchd (Open vSwitch) 
> 2.9.0
> 2018-06-11T09:57:52.812Z|00135|dpdk|ERR|EAL: Unable to find a bus for the device 'mlx4_0'
> 2018-06-11T09:57:52.812Z|00136|netdev_dpdk|WARN|Error attaching device 
> 'mlx4_0' to DPDK
> 2018-06-11T09:57:52.812Z|00137|netdev|WARN|enp59s0d1: could not set 
> configuration (Invalid argument)
> 2018-06-11T09:57:52.825Z|00138|dpdk|ERR|EAL: Unable to find a bus for the device 'mlx4_0'
> 2018-06-11T09:57:52.825Z|00139|netdev_dpdk|WARN|Error attaching device 
> 'mlx4_0' to DPDK
> 2018-06-11T09:57:52.825Z|00140|netdev|WARN|enp59s0d1: could not set 
> configuration (Invalid argument)
> 2018-06-11T09:57:52.832Z|00141|dpdk|ERR|EAL: Unable to find a bus for the device 'mlx4_0'
> 2018-06-11T09:57:52.832Z|00142|netdev_dpdk|WARN|Error attaching device 
> 'mlx4_0' to DPDK
> 2018-06-11T09:57:52.832Z|00143|netdev|WARN|enp59s0d1: could not set 
> configuration (Invalid argument)
> 2018-06-11T09:57:52.843Z|00144|dpdk|ERR|EAL: Unable to find a bus for the device 'mlx4_0'
> 2018-06-11T09:57:52.843Z|00145|netdev_dpdk|WARN|Error attaching device 
> 'mlx4_0' to DPDK
> 2018-06-11T09:57:52.843Z|00146|netdev|WARN|enp59s0d1: could not set 
> configuration (Invalid argument)
> 2018-06-11T09:57:52.856Z|00147|dpdk|ERR|EAL: Unable to find a bus for the device 'mlx4_0'
> 2018-06-11T09:57:52.856Z|00148|netdev_dpdk|WARN|Error attaching device 
> 'mlx4_0' to DPDK
> 2018-06-11T09:57:52.856Z|00149|netdev|WARN|enp59s0d1: could not set 
> configuration (Invalid argument)
> 2018-06-11T09:57:52.862Z|00150|dpdk|ERR|EAL: Unable to find a bus for the device 'mlx4_0'
> 2018-06-11T09:57:52.862Z|00151|netdev_dpdk|WARN|Error attaching device 
> 'mlx4_0' to DPDK
> 2018-06-11T09:57:52.862Z|00152|netdev|WARN|enp59s0d1: could not set 
> configuration (Invalid argument)
> 2018-06-11T09:57:52.870Z|00153|dpdk|ERR|EAL: Unable to find a bus for the device 'mlx4_0'
> 2018-06-11T09:57:52.870Z|00154|netdev_dpdk|WARN|Error attaching device 
> 'mlx4_0' to DPDK
> 2018-06-11T09:57:52.870Z|00155|netdev|WARN|enp59s0d1: could not set 
> configuration (Invalid argument)
> 2018-06-11T09:57:52.889Z|00156|dpdk|ERR|EAL: Unable to find a bus for the device 'mlx4_0'
> 2018-06-11T09:57:52.889Z|00157|netdev_dpdk|WARN|Error attaching device 
> 'mlx4_0' to DPDK
> 2018-06-11T09:57:52.889Z|00158|netdev|WARN|enp59s0d1: could not set 
> configuration (Invalid argument)
> 2018-06-11T09:57:52.903Z|00159|dpdk|ERR|EAL: Unable to find a bus for the device 'mlx4_0'
> 2018-06-11T09:57:52.903Z|00160|netdev_dpdk|WARN|Error attaching device 
> 'mlx4_0' to DPDK
> 2018-06-11T09:57:52.903Z|00161|netdev|WARN|enp59s0d1: could not set 
> configuration (Invalid argument)
> 2018-06-11T09:57:52.908Z|00162|dpdk|ERR|EAL: Unable to find a bus for the device 'mlx4_0'
> 2018-06-11T09:57:52.908Z|00163|netdev_dpdk|WARN|Error attaching device 
> 'mlx4_0' to DPDK
> 2018-06-11T09:57:52.908Z|00164|netdev|WARN|enp59s0d1: could not set 
> configuration (Invalid argument)
> 2018-06-11T09:57:52.914Z|00165|dpdk|ERR|EAL: Unable to find a bus for the device 'mlx4_0'
> 2018-06-11T09:57:52.914Z|00166|netdev_dpdk|WARN|Error attaching device 
> 'mlx4_0' to DPDK
> 2018-06-11T09:57:52.914Z|00167|netdev|WARN|enp59s0d1: could not set 
> configuration (Invalid argument)
> 2018-06-11T09:57:52.923Z|00168|dpdk|ERR|EAL: Unable to find a bus for the device 'mlx4_0'
> 2018-06-11T09:57:52.923Z|00169|netdev_dpdk|WARN|Error attaching device 
> 'mlx4_0' to DPDK
> 2018-06-11T09:57:52.923Z|00170|netdev|WARN|enp59s0d1: could not set 
> configuration (Invalid argument)
> 2018-06-11T09:57:54.808Z|00171|memory|INFO|1406100 kB peak resident 
> set size after 10.1 seconds
> 2018-06-11T09:57:54.808Z|00172|memory|INFO|handlers:118 ofconns:3
> ports:8 revalidators:42 rules:36 udpif keys:1
> 2018-06-11T09:58:02.850Z|00173|connmgr|INFO|br-int<->tcp:127.0.0.1:6633:
> 5 flow_mods 10 s ago (5 adds)
> 2018-06-11T09:58:02.898Z|00174|connmgr|INFO|br-ex<->tcp:127.0.0.1:6633:
> 2 flow_mods 10 s ago (2 adds)
> 2018-06-11T09:58:02.930Z|00175|connmgr|INFO|br-tun<->tcp:127.0.0.1:6633:
> 12 flow_mods 10 s ago (12 adds)
>
> Was that what you mean by init log?
> Best regards
>
> On 06/11/2018 11:51 AM, Raslan Darawsheh wrote:
>> Hi Tobias,
>>
>> Can you try using the following ?
>>
>> ovs-vsctl add-port br-mlx enp59s0d1 -- set Interface enp59s0d1 
>> type=dpdk options:dpdk-devargs=mlx4_0
>>
>> if it didn't work can you send the OVS-DPDK init log ?
>>
>> Kindest regards,
>> Raslan Darawsheh
>>
>> -----Original Message-----
>> From: users [mailto:users-bounces at dpdk.org] On Behalf Of Tobias Urdin
>> Sent: Monday, June 11, 2018 12:15 PM
>> To: users at dpdk.org
>> Subject: [dpdk-users] mellanox connect-x 3 pro dpdk with openvswitch
>>
>> Hello,
>>
>> I'm stuck trying to test DPDK with a Mellanox Connect-X 3 Pro card on 
>> Ubuntu 16.04
>>
>> I've enabled iommu, fixed hugepages and has installed and are running the openvswitch-switch-dpdk alternative.
>>
>> $lspci | grep -i mellanox
>> 3b:00.0 Ethernet controller: Mellanox Technologies MT27520 Family
>> [ConnectX-3 Pro]
>>
>> $lsmod | grep -i mlx
>> mlx4_en               110592  0
>> vxlan                  49152  2 i40e,mlx4_en
>> mlx4_core             286720  1 mlx4_en
>> ptp                    20480  3 igb,i40e,mlx4_en
>>
>> $cat /etc/modprobe.d/mlx4_core.conf
>> options mlx4_core log_num_mgm_entry_size=-1
>>
>>
>> $ovs-vsctl add-port br-mlx dpdk0 -- set Interface dpdk0 type=dpdk
>>
>>     Bridge br-mlx
>>         Port "dpdk0"
>>             Interface "dpdk0"
>>                 type: dpdk
>>                 error: "'dpdk0' is missing 'options:dpdk-devargs'. The old 'dpdk<port_id>' names are not supported"
>>
>>
>> $ovs-vsctl add-port br-mlx enp59s0d1 -- set Interface enp59s0d1 
>> type=dpdk options:dpdk-devargs=0000:3b:00.0,1
>> ovs-vsctl: Error detected while setting up 'enp59s0d1': Error attaching device '0000:3b:00.0,1' to DPDK.  See ovs-vswitchd log for details.
>> ovs-vsctl: The default log directory is "/var/log/openvswitch".
>>
>> 2018-06-11T09:10:38.696Z|00177|netdev_dpdk|WARN|Error attaching 
>> device '0000:3b:00.0,1' to DPDK
>> 2018-06-11T09:10:38.696Z|00178|netdev|WARN|enp59s0d1: could not set 
>> configuration (Invalid argument)
>>
>> 2018-06-11T09:12:50.994Z|00181|dpdk|ERR|EAL: Driver cannot attach the 
>> device (0000:3b:00.0)
>> 2018-06-11T09:12:50.994Z|00182|dpdk|ERR|EAL: No port found for device
>> (0000:3b:00.0) 2018-06-11T09:12:50.994Z|00183|netdev_dpdk|WARN|Error
>> attaching device '0000:3b:00.0,1' to DPDK
>> 2018-06-11T09:12:50.994Z|00184|netdev|WARN|enp59s0d1: could not set 
>> configuration (Invalid argument)
>>
>> $ovs-vsctl add-port br-mlx dpdk0 -- set Interface dpdk0 type=dpdk options:dpdk-devargs="class=eth,mac=ec:0d:9a:b0:13:21"
>> ovs-vsctl: Error detected while setting up 'dpdk0': Error attaching device 'class=eth,mac=ec:0d:9a:b0:13:21' to DPDK.  See ovs-vswitchd log for details.
>> ovs-vsctl: The default log directory is "/var/log/openvswitch".
>>
>>         Port "dpdk0"
>>             Interface "dpdk0"
>>                 type: dpdk
>>                 options: {dpdk-devargs="class=eth,mac=ec:0d:9a:b0:13:21"}
>>                 error: "Error attaching device 'class=eth,mac=ec:0d:9a:b0:13:21' to DPDK"
>>
>>
>> Anybody know what I might have missed? I have tried restarting the openvswitch services after adding the interfaces as well but no difference.
>>
>> The logs doesn't say anything interesting from what I can see :(
>>
>> Best regards
>>
>



More information about the users mailing list