[dpdk-dev] Problem with Intel i40e XL710 dpdk driver

[Karthick, A.R.]

I found DMAR errors while bringing up other ports except port 0.
So rebooting the kernel with intel_iommu=off fixes it and dpdk i40e
initializes fine for all ports.

For some reason, the intel qcu64e mode change utility doesn't work if I
want to change back the mode from 4x10 to 2x40.
Tried several times but never reverts back to 2x40 from 4x10 though that is
a different issue.

Regards,
-Karthick

On Thu, Jan 14, 2016 at 6:26 PM, Karthick, A.R. <kramanar at ciena.com> wrote:

> Hi,
>  I am seeing a "Failed to init adminq: -54" or admin queue timeouts
>  while initializing the admin queue for i40e xl710 intel nic.
>  (Intel server is a E5-2670)
>
>  First things first.
>  I am running the latest firmware.
>  The kernel module is not loaded and yes, it works with the i40e kernel
> driver. (latest or otherwise)
> And this problem comes even with dpdk 2.0/2.1 or the latest stable. So
> there's that.
>
>  I have done a bunch of debugging and here are my findings.
>  With the card configured in 2x40g or 4x10g mode, it _ALWAYS_ works with
>  successfully initializing pci function 0 or port 0.
>  It always fails to subsequently initialize the rest.
>  Even if unbind the igb uio for port 0 and bind only port 1 or port 2,3,4
> in 4x10g mode,
>  it fails.
>
>  Since it works with the kernel driver, I tried to see if there were
> differences in the way registers are setup for i40e driver in kernel and
> dpdk.
>  They look mostly to be the same but obviously there were subtle
> differences.
>  From what I could fathom, I couldn't see much and whatever little was
> caught, I tried to keep the dpdk code in sync and it still failed.
>
>  While stepping through gdb all the way from eal pci to pci uio map to
> eth_i40e_dev_init,
>  to the failure in obtaining the firmware revision for port1 during
> i40e_init_adminq,
>  I did confirm that the memory map was right for the pci.
>
>  So the hw->hw_addr looks correct for port 1 correlating it to the uio1
> map or the physical address from lspci or kernel driver when using the
> kernel driver which works.
>
>  However the admin queue seems to be not processing any request for port 1.
>  Note that port 0 always works and its the same code for others with a
> different eal dev/hw instance.
>
> But for other ports like port1, after correctly setting up the adminq
> registers and memory map,
>  it always fails to obtain the firmware revision since the i40e_asq_done
> is returning 0 for the
>  head register at 0x80300 and doesn't match the next_in_use when starting
> at 1.
>  So it always returns pending or false in i40e_asq_done which is retried a
> certain times after resetting the aq by i40e_init_adminq but ultimately
> gives up.
>
>  Thoughts and wondering if you guys have seen this and have a fix or patch
> that is not in upstream yet.
>
>  Failure enclosed below as mentioned above in detail: (with a 4x10g mode
> for the card but same failure with 2x40g mode as well. No difference. Port
> 0 always succeeds but subsequent ports fail.
>  And same result even with port 0 not bound and starting with the
> initialization of port 2,3,4 which always fails.
>
> EAL: lcore 1 is ready (tid=6bd30700;cpuset=[1])
> EAL: PCI device 0000:01:00.0 on NUMA socket 0
> EAL:   probe driver: 8086:1521 rte_igb_pmd
> EAL:   Not managed by a supported kernel driver, skipped
> EAL: PCI device 0000:01:00.1 on NUMA socket 0
> EAL:   probe driver: 8086:1521 rte_igb_pmd
> EAL:   Not managed by a supported kernel driver, skipped
> EAL: PCI device 0000:83:00.0 on NUMA socket 1
> EAL:   probe driver: 8086:1583 rte_i40e_pmd
> EAL:   PCI memory mapped at 0x7f2f80000000
> EAL:   PCI memory mapped at 0x7f2f80800000
> PMD: eth_i40e_dev_init(): FW 4.40 API 1.4 NVM 04.05.03 eetrack 80001dca
> PMD: i40e_pf_parameter_init(): Max supported VSIs:34
> PMD: i40e_pf_parameter_init(): PF queue pairs:64
> PMD: i40e_pf_parameter_init(): Max VMDQ VSI num:34
> PMD: i40e_pf_parameter_init(): VMDQ queue pairs:4
> EAL: PCI device 0000:83:00.1 on NUMA socket 1
> EAL:   probe driver: 8086:1583 rte_i40e_pmd
> EAL:   PCI memory mapped at 0x7f2f80808000
> EAL:   PCI memory mapped at 0x7f2f81008000
> PMD: eth_i40e_dev_init(): Failed to init adminq: -54
> EAL: Error - exiting with code: 1
>  Cause: Requested device 0000:83:00.1 cannot be used
>
> Regards,
> -Karthick
>
>
>
>