[dpdk-dev] [RFC PATCH] vfio: avoid re-installing irq handler

[Hyong Youb Kim (hyonkim)]

> -----Original Message-----
> From: Jerin Jacob Kollanukkaran <jerinj at marvell.com>
> Sent: Tuesday, July 16, 2019 1:51 AM
> To: Hyong Youb Kim (hyonkim) <hyonkim at cisco.com>; David Marchand
> <david.marchand at redhat.com>; Thomas Monjalon
> <thomas at monjalon.net>; Ferruh Yigit <ferruh.yigit at intel.com>
> Cc: dev at dpdk.org; John Daley (johndale) <johndale at cisco.com>; Shahed
> Shaikh <shshaikh at marvell.com>; Nithin Kumar Dabilpuram
> <ndabilpuram at marvell.com>
> Subject: RE: [RFC PATCH] vfio: avoid re-installing irq handler
> 
> > -----Original Message-----
> > From: Hyong Youb Kim <hyonkim at cisco.com>
> > Sent: Monday, July 15, 2019 9:28 PM
> > To: David Marchand <david.marchand at redhat.com>; Thomas Monjalon
> > <thomas at monjalon.net>; Jerin Jacob Kollanukkaran <jerinj at marvell.com>;
> > Ferruh Yigit <ferruh.yigit at intel.com>
> > Cc: dev at dpdk.org; John Daley <johndale at cisco.com>; Hyong Youb Kim
> > <hyonkim at cisco.com>
> > Subject: [EXT] [RFC PATCH] vfio: avoid re-installing irq handler
> >
> > A rough patch for the approach mentioned earlier. It is only for discussion.
> > http://mails.dpdk.org/archives/dev/2019-July/138113.html
> >
> > To try this out, first revert the following then apply.
> > commit 89aac60e0be9 ("vfio: fix interrupts race condition")
> 
> Yes. This patch has to be to reverted. It changes the existing interrupt
> behavior and does not address the MSIX case as well.
> 
> I think, The clean fix would be to introduce rte_intr_mask() and
> rte_intr_unmask() by abstracting the INTX and MSIX differences
> And let qede driver call it as needed.
> 
> Thoughts?

Hi,

You are proposing these?
- Add rte_intr_mask_intx, rte_intr_unmask_intx.
  No APIs for masking MSI/MSI-X as vfio-pci does not support that.
- Modify PMD irq handlers to use rte_intr_unmask_intx as necessary.

That might be too intrusive. And too much work for the sake of
INTx.. Anyone really using/needing INTx these days? :-)

The following drivers call rte_intr_enable from their irq handlers. So
with explicit rte_intr_unmask_intx, all these would need to do "if
using intx, unmask"?

atlantic, avp, axgbe, bnx2x, e1000, fm10k, ice, ixgbe, nfp, qede, sfc,
vmxnet3

And nfp seems to rely on rte_intr_enable to re-install irq handler to
unmask a vector in MSI-X Table?

        if (hw->ctrl & NFP_NET_CFG_CTRL_MSIXAUTO) {
                /* If MSI-X auto-masking is used, clear the entry */
                rte_wmb();
                rte_intr_enable(&pci_dev->intr_handle);

With David's patch and mine, this handler would have to first
rte_intr_disable() and then enable, if such unmasking is really
necessary..

As for the semantics of rte_intr_enable/disable, I am ok as is.
- "enable": put things in a state where NIC can send an interrupt, and
  PMD/app gets a callback.
  Whether this involves unmasking for INTx is hidden.
- "disable": put things in a state where NIC cannot send an interrupt.

Regardless of vfio changes, we should probably remove rte_intr_enable
from qede_interrupt_handler (the MSI/MSI-X interrupt handler), to make
usage/intention clear..

Thanks.
-Hyong