[dpdk-dev] vf init issue with patch igb_uio: issue FLR during open and release of device file

Gregory Etelson gregory at weka.io
Sun Sep 17 04:49:41 CEST 2017


Hello Qiming,

Did you try to remove PCI reset at igbuio_pci_open ?
Reverting the patch entirely will get us back to VF issues

Regards,
Gregory 


On Friday, 15 September 2017 11:04:16 IDT Yang, Qiming wrote:
> Hi, Thomas
> We meet a urgent issue because of Shijith's patch. 
> When use DPDK version 17.08, DPDK VF driver and kernel PF driver, i40e, ixgbe and qede(Harish said) VF all meet initial failure.
> This issue only occurred when use 17.08 igb_uio, vfio-pci and 17.05 igb_uio all works well.
> Considering this issue will block our vf driver develop and test work, could we revert this patch first and find another appropriate way?
> 
> Qiming
> 
> > -----Original Message-----
> > From: Patil, Harish [mailto:Harish.Patil at cavium.com]
> > Sent: Thursday, September 14, 2017 3:45 AM
> > To: Yigit, Ferruh <ferruh.yigit at intel.com>; Hu, Xuekun <xuekun.hu at intel.com>;
> > Thotton, Shijith <Shijith.Thotton at cavium.com>; Yang, Qiming
> > <qiming.yang at intel.com>; Gregory Etelson <gregory at weka.io>
> > Cc: dev at dpdk.org; Tan, Jianfeng <jianfeng.tan at intel.com>
> > Subject: Re: [dpdk-dev] vf init issue with patch igb_uio: issue FLR during open
> > and release of device file
> > 
> > 
> > 
> > -----Original Message-----
> > From: dev <dev-bounces at dpdk.org> on behalf of Ferruh Yigit
> > <ferruh.yigit at intel.com>
> > Date: Wednesday, September 13, 2017 at 10:06 AM
> > To: "Hu, Xuekun" <xuekun.hu at intel.com>, "Thotton, Shijith"
> > <Shijith.Thotton at cavium.com>, "Yang, Qiming" <qiming.yang at intel.com>,
> > Gregory Etelson <gregory at weka.io>
> > Cc: "dev at dpdk.org" <dev at dpdk.org>, "Tan, Jianfeng" <jianfeng.tan at intel.com>
> > Subject: Re: [dpdk-dev] vf init issue with patch igb_uio: issue FLR during open
> > and release of device file
> > 
> > >On 9/13/2017 3:25 PM, Hu, Xuekun wrote:
> > >> I met the same issue too, only with i40e 2.1.26 PF kernel driver.
> > >>
> > >> -----Original Message-----
> > >> From: dev [mailto:dev-bounces at dpdk.org] On Behalf Of Ferruh Yigit
> > >> Sent: Wednesday, September 13, 2017 7:04 PM
> > >> To: Shijith Thotton <shijith.thotton at caviumnetworks.com>; Yang,
> > >>Qiming <qiming.yang at intel.com>; Gregory Etelson <gregory at weka.io>
> > >> Cc: dev at dpdk.org; Tan, Jianfeng <jianfeng.tan at intel.com>
> > >> Subject: Re: [dpdk-dev] vf init issue with patch igb_uio: issue FLR
> > >>during open and release of device file
> > >>
> > >> On 9/13/2017 11:48 AM, Shijith Thotton wrote:
> > >>> On Wed, Sep 13, 2017 at 07:51:30AM +0000, Yang, Qiming wrote:
> > >>>>    Hi, Shijith
> > >>>>
> > >>>>
> > >>>>
> > >>>>    VF init error will happen after apply your patch, error log as
> > >>>>below. If
> > >>>>    revert your commit, all things work well. And this issue is not
> > >>>>only occur
> > >>>>    in i40 VF but also ixgbe.
> > >
> > >Hi Qiming,
> > >
> > >I can reproduce the issue for the case:
> > >
> > >1- i40e 2.1.26 PF kernel driver (NOT reproduced with 1.6.27-k)
> > >2- DPDK app run on host, so both PF and VF are in host. Linux driver
> > >for PF and DPDK for VF. (When VF used in a VM, Linux PF on host and
> > >DPDK VF on guest, this works fine)
> > >
> > >And I confirm the function igbuio_pci_open() added with below patch is
> > >causing it.
> > >
> > >igbuio_pci_open() does:
> > >    pci_reset_function(dev);
> > >    pci_set_master(dev);
> > >
> > >Don't know yet root cause of the error, will dig more.
> > >
> > >>>>
> > >>>>    Could you help to check it soon?
> > >>>>
> > >>>>
> > >>>>
> > >>>>    [root at localhost app]# ./testpmd -c 7 -n 4 -- -i
> > >>>>
> > >>>>    EAL: Detected 10 lcore(s)
> > >>>>
> > >>>>    EAL: No free hugepages reported in hugepages-1048576kB
> > >>>>
> > >>>>    EAL: Probing VFIO support...
> > >>>>
> > >>>>    EAL: WARNING: cpu flags constant_tsc=yes nonstop_tsc=no -> using
> > >>>>    unreliable clock cycles !
> > >>>>
> > >>>>    EAL: PCI device 0000:00:03.0 on NUMA socket -1
> > >>>>
> > >>>>    EAL:   Invalid NUMA socket, default to 0
> > >>>>
> > >>>>    EAL:   probe driver: 8086:154c net_i40e_vf
> > >>>>
> > >>>>    i40evf_init_vf(): init_adminq failed: -53
> > >>>>
> > >>>>    i40evf_dev_init(): Init vf failed
> > >>>>
> > >>>>    EAL: Requested device 0000:00:03.0 cannot be used
> > >>>>
> > >>>>
> > >>>>
> > >>>>    commit b58eedfc7dd57eef6d12e2c654a52c834f36084a
> > >>>>    Author: Shijith Thotton <shijith.thotton at caviumnetworks.com>
> > >>>>    Date: Fri Jul 7 16:43:51 2017 +0530
> > >>>>
> > >>>>    igb_uio: issue FLR during open and release of device file
> > >>>>
> > >>>>    Set UIO info device file operations open and release. Call pci
> > >>>>reset
> > >>>>    function inside open and release to clear device state at start
> > >>>>and end.
> > >>>>    Copied this behaviour from vfio_pci kernel module code. With
> > >>>>this patch,
> > >>>>    it is not mandatory to issue FLR by PMD's during init and close.
> > >>>>
> > >>>>    Bus master enable and disable are added in open and release
> > >>>>respectively
> > >>>>    to take care of device DMA.
> > >>>>
> > >>>>    Signed-off-by: Shijith Thotton <shijith.thotton at caviumnetworks.com>
> > >>>>    Reviewed-by: Jianfeng Tan <jianfeng.tan at intel.com>
> > >>>>    Acked-by: Ferruh Yigit <ferruh.yigit at intel.com>
> > >>>>    Acked-by: Gregory Etelson <gregory at weka.io>
> > >>>>
> > >>>>
> > >>>>
> > >>>>    Best Regard,
> > >>>>
> > >>>>    Yang Qiming
> > >>>>
> > >>>>
> > >>>
> > >>> Hi Yang Qiming,
> > >>>
> > >>> We didn't face this issue while preparing patch. I think, Gregory
> > >>> tested it for Intel NICs. Adding more people to the loop for help.
> > >>>
> > >>
> > >> I will try to reproduce the issue.
> > >>
> > >>> Shijith
> > 
> > [Harish] We are seeing similar issue with the bringup of qede VF devices due to
> > this patch.
> > Could we revert this and leave it to individual PMDs to manage it?
> > >>
> 
> 



More information about the dev mailing list