[dpdk-dev] [sriov] [ixgbevf] We have to down PF and VF port to reconfig a port?

Sam batmanustc at gmail.com
Fri Jan 5 04:27:09 CET 2018


This is kernel ixgbevf error log:

Dec 27 20:23:22 yf-mos-test-net14 kernel: [103406.865542] ------------[ cut
here ]------------
Dec 27 20:23:22 yf-mos-test-net14 kernel: [103406.865556] WARNING: at
net/sched/sch_generic.c:297 dev_watchdog+0x276/0x280()
Dec 27 20:23:22 yf-mos-test-net14 kernel: [103406.865558] NETDEV WATCHDOG:
enp1s16f1 (ixgbevf): transmit queue 0 timed out
Dec 27 20:23:22 yf-mos-test-net14 kernel: [103406.865559] Modules linked
in: nbd iptable_mangle iptable_nat nf_conntrack_ipv4 nf_defrag_ipv4
nf_nat_ipv4 nf_nat nf_con
ntrack iptable_filter vhost_net vhost macvtap macvlan ixgbevf(OE)
igb_uio(OE) udp_diag unix_diag af_packet_diag netlink_diag tun uio tcp_diag
inet_diag bonding ext4 m
bcache jbd2 intel_powerclamp coretemp kvm_intel kvm irqbypass crc32_pclmul
ghash_clmulni_intel aesni_intel lrw gf128mul glue_helper ablk_helper cryptd
ipmi_devintf sg
 iTCO_wdt iTCO_vendor_support dcdbas pcspkr mei_me mei lpc_ich sb_edac
edac_core ipmi_si wmi shpchp ipmi_msghandler acpi_pad acpi_power_meter
ip_tables xfs libcrc32c
sd_mod crc_t10dif mgag200 drm_kms_helper syscopyarea crct10dif_pclmul
sysfillrect crct10dif_common sysimgblt crc32c_intel fb_sys_fops ttm ahci
drm ixgbe libahci igb l
ibata i2c_algo_bit mdio ptp i2c_core megaraid_sas pps_core dca fjes [last
unloaded: nbd]
Dec 27 20:23:22 yf-mos-test-net14 kernel: [103406.865619] CPU: 0 PID: 0
Comm: swapper/0 Tainted: G           OE  ------------
 3.10.0-514.10.2.el7.mt20170307.x86_64
#1
Dec 27 20:23:22 yf-mos-test-net14 kernel: [103406.865622] Hardware name:
Dell Inc. PowerEdge R720/068CDY, BIOS 2.5.2 01/28/2015
Dec 27 20:23:22 yf-mos-test-net14 kernel: [103406.865624]  ffff880fff803d88
d4910673413043a2 ffff880fff803d40 ffffffff8168648f
Dec 27 20:23:22 yf-mos-test-net14 kernel: [103406.865626]  ffff880fff803d78
ffffffff81085940 0000000000000000 ffff880ffdfd0000
Dec 27 20:23:22 yf-mos-test-net14 kernel: [103406.865629]  ffff880ffdfd1940
0000000000000008 0000000000000000 ffff880fff803de0
Dec 27 20:23:22 yf-mos-test-net14 kernel: [103406.865631] Call Trace:
Dec 27 20:23:22 yf-mos-test-net14 kernel: [103406.865632]  <IRQ>
[<ffffffff8168648f>] dump_stack+0x19/0x1b
Dec 27 20:23:22 yf-mos-test-net14 kernel: [103406.865639]
[<ffffffff81085940>] warn_slowpath_common+0x70/0xb0
Dec 27 20:23:22 yf-mos-test-net14 kernel: [103406.865641]
[<ffffffff810859dc>] warn_slowpath_fmt+0x5c/0x80
Dec 27 20:23:22 yf-mos-test-net14 kernel: [103406.865644]
[<ffffffff81596616>] dev_watchdog+0x276/0x280
Dec 27 20:23:22 yf-mos-test-net14 kernel: [103406.865647]
[<ffffffff815963a0>] ? dev_graft_qdisc+0x80/0x80
Dec 27 20:23:22 yf-mos-test-net14 kernel: [103406.865650]
[<ffffffff81095b46>] call_timer_fn+0x36/0x110
Dec 27 20:23:22 yf-mos-test-net14 kernel: [103406.865652]
[<ffffffff815963a0>] ? dev_graft_qdisc+0x80/0x80
Dec 27 20:23:22 yf-mos-test-net14 kernel: [103406.865655]
[<ffffffff81098837>] run_timer_softirq+0x237/0x340
Dec 27 20:23:22 yf-mos-test-net14 kernel: [103406.865658]
[<ffffffff8108f2cf>] __do_softirq+0xef/0x280
Dec 27 20:23:22 yf-mos-test-net14 kernel: [103406.865661]
[<ffffffff8169851c>] call_softirq+0x1c/0x30
Dec 27 20:23:22 yf-mos-test-net14 kernel: [103406.865664]
[<ffffffff8102d365>] do_softirq+0x65/0xa0
Dec 27 20:23:22 yf-mos-test-net14 kernel: [103406.865667]
[<ffffffff8108f665>] irq_exit+0x115/0x120
Dec 27 20:23:22 yf-mos-test-net14 kernel: [103406.865670]
[<ffffffff81699195>] smp_apic_timer_interrupt+0x45/0x60
Dec 27 20:23:22 yf-mos-test-net14 kernel: [103406.865672]
[<ffffffff816976dd>] apic_timer_interrupt+0x6d/0x80
Dec 27 20:23:22 yf-mos-test-net14 kernel: [103406.865673]  <EOI>
[<ffffffff810b4776>] ? hrtimer_start_range_ns+0x1c6/0x390
Dec 27 20:23:22 yf-mos-test-net14 kernel: [103406.865679]
[<ffffffff81514392>] ? cpuidle_enter_state+0x52/0xc0
Dec 27 20:23:22 yf-mos-test-net14 kernel: [103406.865681]
[<ffffffff81514388>] ? cpuidle_enter_state+0x48/0xc0
Dec 27 20:23:22 yf-mos-test-net14 kernel: [103406.865683]
[<ffffffff815144d9>] cpuidle_idle_call+0xd9/0x210
Dec 27 20:23:22 yf-mos-test-net14 kernel: [103406.865687]
[<ffffffff810350ee>] arch_cpu_idle+0xe/0x30
Dec 27 20:23:22 yf-mos-test-net14 kernel: [103406.865690]
[<ffffffff810e7e65>] cpu_startup_entry+0x245/0x290
Dec 27 20:23:22 yf-mos-test-net14 kernel: [103406.865694]
[<ffffffff816748e7>] rest_init+0x77/0x80
Dec 27 20:23:22 yf-mos-test-net14 kernel: [103406.865698]
[<ffffffff81b0a05a>] start_kernel+0x42c/0x44d
Dec 27 20:23:22 yf-mos-test-net14 kernel: [103406.865700]
[<ffffffff81b09a37>] ? repair_env_string+0x5c/0x5c
Dec 27 20:23:22 yf-mos-test-net14 kernel: [103406.865703]
[<ffffffff81b09120>] ? early_idt_handlers+0x120/0x120
Dec 27 20:23:22 yf-mos-test-net14 kernel: [103406.865705]
[<ffffffff81b095ee>] x86_64_start_reservations+0x2a/0x2c
Dec 27 20:23:22 yf-mos-test-net14 kernel: [103406.865708]
[<ffffffff81b09742>] x86_64_start_kernel+0x152/0x175
Dec 27 20:23:22 yf-mos-test-net14 kernel: [103406.865709] ---[ end trace
45a157a1f564e086 ]---



2017-12-27 10:54 GMT+08:00 Sam <batmanustc at gmail.com>:

>
> 2017-12-27 10:42 GMT+08:00 Sam <batmanustc at gmail.com>:
>
>> Hi all,
>>
>> I'm running OVS-DPDK on a two netdevice server(eth0 and eth1), and I use
>> sriov to make one VF and one PF on each netdevice, as bellow. Please omit
>> eth2 and eth3. Then two VF port(enp1s16 and enp1s16f1) combine into linux
>> bond1 port as management port, two PF port(0000:01:00.0 and 0000:01:00.1)
>> combine into OVS-DPDK bond port(mode 4) as dataplane port.
>>
>> [root at yf-mos-test-net14 ~]# /usr/local/share/openvswitch/scripts/dpdk_nic_bind
>>> --status
>>> Network devices using DPDK-compatible driver
>>> ============================================
>>> 0000:01:00.0 'Ethernet Controller 10-Gigabit X540-AT2' drv=igb_uio
>>> unused=ixgbe
>>> 0000:01:00.1 'Ethernet Controller 10-Gigabit X540-AT2' drv=igb_uio
>>> unused=ixgbe
>>> Network devices using kernel driver
>>> ===================================
>>> 0000:01:10.0 'X540 Ethernet Controller Virtual Function' if=enp1s16
>>> drv=ixgbevf unused=igb_uio
>>> 0000:01:10.1 'X540 Ethernet Controller Virtual Function' if=enp1s16f1
>>> drv=ixgbevf unused=igb_uio
>>> 0000:08:00.0 'I350 Gigabit Network Connection' if=eth2 drv=igb
>>> unused=igb_uio
>>> 0000:08:00.1 'I350 Gigabit Network Connection' if=eth3 drv=igb
>>> unused=igb_uio
>>> Other network devices
>>> =====================
>>> <none>
>>
>>
>> I change OVS-DPDK code, and use command to stop  OVS-DPDK bond port and
>> its slaves, and then change its txconf and rxconf, and then start up them.
>> But the kernel report "Detected Tx Unit Hang ..." and no packets could go
>> through OVS-DPDK bond port and linux bond port, but use command to show
>> OVS-DPDK port and linux bond port,
>>
> all status is NOT OK(LACP status not OK, I don't if it's because link
> error or LACP packet process error).
>
>>
>> But when I stop linux bond port first, and then do the same process as
>> above, no error report and packets could go through OVS-DPDK bond port and
>> linux bond port.
>>
>> So what's this problem and how to fix it ? Thank you~
>>
> I want to make OVS-DPDK bond port could be stop/start/reconfig while
> running. If others use some other way to do this, could you share it, thank
> you~
>


More information about the dev mailing list