<div dir="ltr"><div>Hi Stephen,</div><div><br></div>ixgbe interfaces works correctly when not used with dpdk. <div><br><div># cat /proc/interrupts | grep enp2s0f1<br>109: 436 0 0 0 IR-PCI-MSI 1050624-edge enp2s0f1-TxRx-0<br>110: 0 167 0 0 IR-PCI-MSI 1050625-edge enp2s0f1-TxRx-1<br>111: 0 0 306 0 IR-PCI-MSI 1050626-edge enp2s0f1-TxRx-2<br>112: 0 0 0 173 IR-PCI-MSI 1050627-edge enp2s0f1-TxRx-3<br>113: 0 0 1 0 IR-PCI-MSI 1050628-edge enp2s0f1<br></div><div><br></div><div>dmesg doesn't seem to offer any clue.</div><div><br></div><div># dmesg | grep ixgbe<br>[ 7.680989] ixgbe: Intel(R) 10 Gigabit PCI Express Network Driver<br>[ 7.680996] ixgbe: Copyright (c) 1999-2016 Intel Corporation.<br>[ 8.318606] ixgbe 0000:02:00.0: Multiqueue Enabled: Rx Queue count = 4, Tx Queue count = 4 XDP Queue count = 0<br>[ 8.477642] ixgbe 0000:02:00.0: MAC: 6, PHY: 27, PBA No: 000700-000<br>[ 8.477654] ixgbe 0000:02:00.0: 00:e0:ed:db:33:69<br>[ 8.539077] ixgbe 0000:02:00.0: Intel(R) 10 Gigabit Network Connection<br>[ 8.888011] ixgbe 0000:02:00.1: Multiqueue Enabled: Rx Queue count = 4, Tx Queue count = 4 XDP Queue count = 0<br>[ 9.019755] ixgbe 0000:02:00.1: MAC: 6, PHY: 27, PBA No: 000700-000<br>[ 9.019760] ixgbe 0000:02:00.1: 00:e0:ed:db:33:68<br>[ 9.063320] ixgbe 0000:02:00.1: Intel(R) 10 Gigabit Network Connection<br></div><div><br></div><div>Thanks & Regards,</div><div>Rajasekhar</div></div></div><br><div class="gmail_quote"><div dir="ltr" class="gmail_attr">On Wed, Mar 8, 2023 at 11:14 PM Stephen Hemminger <<a href="mailto:stephen@networkplumber.org">stephen@networkplumber.org</a>> wrote:<br></div><blockquote class="gmail_quote" style="margin:0px 0px 0px 0.8ex;border-left:1px solid rgb(204,204,204);padding-left:1ex">On Wed, 8 Mar 2023 22:54:12 +0530<br>
Rajasekhar Pulluru <<a href="mailto:pullururajasekhar@gmail.com" target="_blank">pullururajasekhar@gmail.com</a>> wrote:<br>
<br>
> No Honnappa.<br>
> <br>
> Thanks & Regards,<br>
> Rajasekhar<br>
> <br>
> On Wed, Mar 8, 2023 at 5:49 AM Honnappa Nagarahalli <<br>
> <a href="mailto:Honnappa.Nagarahalli@arm.com" target="_blank">Honnappa.Nagarahalli@arm.com</a>> wrote: <br>
> <br>
> ><br>
> ><br>
> > From: Rajasekhar Pulluru <<a href="mailto:pullururajasekhar@gmail.com" target="_blank">pullururajasekhar@gmail.com</a>><br>
> > Sent: Tuesday, March 7, 2023 12:52 PM<br>
> > To: <a href="mailto:dev@dpdk.org" target="_blank">dev@dpdk.org</a><br>
> > Subject: ixgbe rxq interrupt not working<br>
> ><br>
> > Hi Team,<br>
> ><br>
> > Bringing-up dpdk-22.07 on an intel machine with 8 ports, 4 of them driven<br>
> > by igb and the rest of the 4 ports driven by ixgbe.<br>
> > [Honnappa] Do you have packets crossing between the 2 drivers?<br>
> ><br>
> ><br>
> > I am following the below sequence to initialize these ports:<br>
> ><br>
> > dev_conf.intr_conf.lsc = 1; //Enable link state change interrupt<br>
> > dev_conf.intr_conf.rxq = 1; //Enable RX Queue Interrupt<br>
> > dev_conf.rxmode.mq_mode = RTE_ETH_MQ_RX_NONE;<br>
> > dev_conf.rxmode.offloads = 0;<br>
> > dev_conf.txmode.mq_mode = RTE_ETH_MQ_TX_NONE;<br>
> > dev_conf.txmode.offloads = 0;<br>
> ><br>
> > rte_eth_dev_configure<br>
> > rte_eth_rx_queue_setup<br>
> > rte_eth_tx_queue_setup<br>
> > rte_eth_dev_start<br>
> > data = port_id << CHAR_BIT | queue_id;<br>
> > rte_eth_dev_rx_intr_ctl_q(port_id, queue_id, RTE_EPOLL_PER_THREAD,<br>
> > RTE_INTR_EVENT_ADD, (void *)((uintptr_t)data));<br>
> > rte_eth_dev_rx_intr_enable(port_id, queue_id);<br>
> ><br>
> > And then main loop repeats the below:<br>
> ><br>
> > rte_epoll_wait(RTE_EPOLL_PER_THREAD, event, 1, timeout /* 200micro-sec<br>
> > */); /* ignore return value */<br>
> > rte_eth_dev_rx_intr_disable(port_id, queue_id);<br>
> > rte_eth_rx_burst(port_id, queue_id, pkts, num_pkts);<br>
> > rte_eth_dev_rx_intr_enable(port_id, queue_id);<br>
> ><br>
> > The code is same for all the ports, igb ports are able to come-up and rx<br>
> > packets, where-as the ixgbe ports are not able to rx packets at all.<br>
> > cat /proc/interrupts dumps vfio-msix counters for ixgbe as 0, where-as<br>
> > it's non-zero for igb.<br>
> > If I don't use/enable rxq interrupt for ixgbe (and remove epoll wait,<br>
> > interrupt enable/disable from while loop) and simply poll for<br>
> > rte_eth_rx_burst in a loop, ixgbe ports are able to rx packets.<br>
> ><br>
> > What could be wrong here? Appreciate any help.<br>
> ><br>
> > I would also like to know if there's an asynchronous rxq interrupt<br>
> > notification to the application instead of rte_epoll_wait (and sleep).<br>
> ><br>
> > Thanks & Regards,<br>
> > Rajasekhar<br>
> > <br>
<br>
Does the device work as expected when not used with DPDK?<br>
I.e does the kernel driver handle it correctly.<br>
<br>
<br>
Also check the kernel dmesg log, for any relevant info.<br>
There maybe VFIO or other overlap involved.<br>
</blockquote></div>