[PATCH v1] event/dlb2: add dequeue interrupt mode support

[Pathak, Pravin]

> -----Original Message-----
> From: Mattias Rönnblom <hofors at lysator.liu.se>
> Sent: Tuesday, July 1, 2025 4:16 AM
> To: Pathak, Pravin <pravin.pathak at intel.com>; Jerin Jacob
> <jerinjacobk at gmail.com>
> Cc: dev at dpdk.org; jerinj at marvell.com; Chen, Mike Ximing
> <mike.ximing.chen at intel.com>; Richardson, Bruce
> <bruce.richardson at intel.com>; thomas at monjalon.net; Marchand, David
> <david.marchand at redhat.com>; nipun.gupta at amd.com;
> chenbox at nvidia.com; Sarkar, Tirthendu <tirthendu.sarkar at intel.com>; Pavan
> Nikhilesh <pbhagavatula at marvell.com>; Shijith Thotton
> <sthotton at marvell.com>; Hemant Agrawal <hemant.agrawal at nxp.com>;
> Sachin Saxena <sachin.saxena at oss.nxp.com>; harry.chang at intel.com;
> Mattias Rönnblom <mattias.ronnblom at ericsson.com>
> Subject: Re: [PATCH v1] event/dlb2: add dequeue interrupt mode support
> 
> On 2025-06-30 19:34, Pathak, Pravin wrote:
> >
> >
> >> -----Original Message-----
> >> From: Mattias Rönnblom <hofors at lysator.liu.se>
> >> Sent: Monday, June 30, 2025 12:51 PM
> >> To: Pathak, Pravin <pravin.pathak at intel.com>; Jerin Jacob
> >> <jerinjacobk at gmail.com>
> >> Cc: dev at dpdk.org; jerinj at marvell.com; Chen, Mike Ximing
> >> <mike.ximing.chen at intel.com>; Richardson, Bruce
> >> <bruce.richardson at intel.com>; thomas at monjalon.net; Marchand, David
> >> <david.marchand at redhat.com>; nipun.gupta at amd.com;
> chenbox at nvidia.com;
> >> Sarkar, Tirthendu <tirthendu.sarkar at intel.com>; Pavan Nikhilesh
> >> <pbhagavatula at marvell.com>; Shijith Thotton <sthotton at marvell.com>;
> >> Hemant Agrawal <hemant.agrawal at nxp.com>; Sachin Saxena
> >> <sachin.saxena at oss.nxp.com>; harry.chang at intel.com; Mattias Rönnblom
> >> <mattias.ronnblom at ericsson.com>
> >> Subject: Re: [PATCH v1] event/dlb2: add dequeue interrupt mode
> >> support
> >>
> >> On 2025-06-30 18:18, Pathak, Pravin wrote:
> >>>
> >>>
> >>>> -----Original Message-----
> >>>> From: Jerin Jacob <jerinjacobk at gmail.com>
> >>>> Sent: Monday, June 30, 2025 7:44 AM
> >>>> To: Mattias Rönnblom <hofors at lysator.liu.se>
> >>>> Cc: Pathak, Pravin <pravin.pathak at intel.com>; dev at dpdk.org;
> >>>> jerinj at marvell.com; Chen, Mike Ximing <mike.ximing.chen at intel.com>;
> >>>> Richardson, Bruce <bruce.richardson at intel.com>;
> >>>> thomas at monjalon.net; Marchand, David
> <david.marchand at redhat.com>;
> >> nipun.gupta at amd.com;
> >>>> chenbox at nvidia.com; Sarkar, Tirthendu <tirthendu.sarkar at intel.com>;
> >>>> Pavan Nikhilesh <pbhagavatula at marvell.com>; Shijith Thotton
> >>>> <sthotton at marvell.com>; Hemant Agrawal
> <hemant.agrawal at nxp.com>;
> >>>> Sachin Saxena <sachin.saxena at oss.nxp.com>; harry.chang at intel.com;
> >>>> Mattias Rönnblom <mattias.ronnblom at ericsson.com>
> >>>> Subject: Re: [PATCH v1] event/dlb2: add dequeue interrupt mode
> >>>> support
> >>>>
> >>>> On Mon, Jun 30, 2025 at 4:47 PM Mattias Rönnblom
> >>>> <hofors at lysator.liu.se>
> >>>> wrote:
> >>>>>
> >>>>> On 2025-06-30 11:19, Jerin Jacob wrote:
> >>>>>> On Sat, Jun 28, 2025 at 11:17 AM Pravin Pathak
> >>>> <pravin.pathak at intel.com> wrote:
> >>>>>>>
> >>>>>>> DLB2 port interrupt is implemented using DPDK interrupt framework.
> >>>>>>> This allows eventdev dequeue API to sleep when the port queue is
> >>>>>>> empty and gets wakeup when event arrives at the port. Port
> >>>>>>> dequeue mode is configured using devargs argument
> port_dequeue_wait.
> >>>>>>> Supported modes are polling and interrupt. Default mode is polling.
> >>>>>>> This commit also adds code to handle device error interrupts and
> >>>>>>> print alarm details.
> >>>>>>>
> >>>>>>> Signed-off-by: Pravin Pathak <pravin.pathak at intel.com>
> >>>>>>> Signed-off-by: Tirthendu Sarkar <tirthendu.sarkar at intel.com>
> >>>>>>> ---
> >>>>>>>     doc/guides/eventdevs/dlb2.rst              |  20 +
> >>>>>>>     drivers/event/dlb2/dlb2.c                  | 236 +++++-
> >>>>>>>     drivers/event/dlb2/dlb2_iface.c            |   7 +
> >>>>>>>     drivers/event/dlb2/dlb2_iface.h            |   8 +
> >>>>>>>     drivers/event/dlb2/dlb2_priv.h             |  18 +
> >>>>>>>     drivers/event/dlb2/dlb2_user.h             | 112 +++
> >>>>>>>     drivers/event/dlb2/pf/base/dlb2_hw_types.h |  70 ++
> >>>>>>>     drivers/event/dlb2/pf/base/dlb2_osdep.h    |  46 ++
> >>>>>>>     drivers/event/dlb2/pf/base/dlb2_regs.h     | 149 +++-
> >>>>>>>     drivers/event/dlb2/pf/base/dlb2_resource.c | 825
> >>>> +++++++++++++++++++++
> >>>>>>>     drivers/event/dlb2/pf/base/dlb2_resource.h |   6 +
> >>>>>>>     drivers/event/dlb2/pf/dlb2_pf.c            | 223 ++++++
> >>>>>>>     12 files changed, 1711 insertions(+), 9 deletions(-)
> >>>>>>>
> >>>>>>> diff --git a/doc/guides/eventdevs/dlb2.rst
> >>>>>>> b/doc/guides/eventdevs/dlb2.rst index 8ec7168f20..a4ba857351
> >>>> 100644
> >>>>>>> --- a/doc/guides/eventdevs/dlb2.rst
> >>>>>>> +++ b/doc/guides/eventdevs/dlb2.rst
> >>>>>>> @@ -477,6 +477,26 @@ Example command to use as meson option
> >> for
> >>>> credit handling:
> >>>>>>>
> >>>>>>>            meson configure -Dc_args='-DDLB_SW_CREDITS_CHECKS=0 -
> >>>> DDLB_HW_CREDITS_CHECKS=1'
> >>>>>>>
> >>>>>>> +Interrupt Mode Support
> >>>>>>> +~~~~~~~~~~~~~~~~~~~~~~
> >>>>>>> +DLB dequeue supports interrupt mode for the API
> >>>> rte_event_dequeue_burst().
> >>>>>>> +The default port dequeue mode is polling. Dequeue wait mode can
> >>>>>>> +be configured on per eventdev port basis using devargs argument
> >>>>>>> +'port_dequeue_wait'. In interrupt mode, if the port queue is
> >>>>>>> +empty, the application thread will block on the interrupt until
> >>>>>>> +a new event arrives. It enters blocking mode only after any
> >>>>>>> +specified timeout. During the timeout, it will poll the port
> >>>>>>> +queue for
> >>>> events as usual. Interrupt mode uses the DPDK interrupt support
> >> framework.
> >>>>>>> +
> >>>>>>> +    .. code-block:: console
> >>>>>>> +
> >>>>>>> +       --allow ea:00.0,port_dequeue_wait=all:interrupt
> >>>>>>
> >>>>>> Adding other eventdev PMD mainatainers.
> >>>>>>
> >>>>>> Looks like it can be a generic feature. i.e set this option is
> >>>>>> dev_configure() If there is no objection, Please send a new patch
> >>>>>> around
> >>>> that.
> >>>>>
> >>>>> I've considered implementing this in DSW, although in a different
> >>>>> manner (with eventfds and poll()).
> >>>>>
> >>>>> The dequeue timeout will still be honored in "interrupt mode", correct?
> >>>>> It wasn't obvious from the description.
> >>>>
> >>>> How is it in Intel PMD?
> >>>
> >>> It would be best if we configure it per port using
> >>> RTE_EVENT_PORT_CFG_*
> >> flags. Will it be, OK?
> >>> The dequeue timeout will be honored, and the decision to block or
> >>> return
> >> will be made after the timeout.
> >>
> >> That doesn't sound like the timeout is honored.
> >>
> >> The reason an application wants the dequeue call to complete within a
> >> certain time, even though there wasn't any events, is likely because
> >> it want to go do something else with that thread, after the timeout.
> >>
> >> Thus, you can't decide to block the thread *after* the timeout. If
> >> you block, you have to do some time *before* the timeout, and wake up
> >> in time to meet the deadline.
> >>
> >> For example, if the event device is given a 1 ms dequeue timeout by
> >> the application, it could go busy-poll for 10 us, then busy-poll with
> >> a short tpause for 100 us, and then put the thread to sleep blocking
> >> on some fd for the remaining 890 us.
> >>
> >> The hardware-specific nature of the timing and exact mechanism to use
> >> speaks against having this kind of configuration in the Eventdev API.
> >
> > The mode is not for what to do during a timeout, but after a timeout.
> 
> My comments were not so much concerning any DLB-specific extension, but
> rather how this kind of function should work, if it was a part of the standard
> API.
> 
> > We can enter sleep mode immediately by setting the timeout to 0 if we need
> to.
> > This mode is not changing the current timeout behavior.  After the
> > timeout, it allows HW devices supporting interrupts to block on an interrupt.
> 
> The current API specifies that control is returned to the application, after the
> timeout has expired. If you change that with a PMD parameter, the DLB
> behavior will be in violation of the API contract. Applications using
> rte_keepalive_alive() between dequeues is one example of those that will
> break. In fact, all applications that use more than one RTE service will break.
> 
> > It will wake up only after a new event arrives at the port. If the
> > application needs control back then it can use the current default
> > non-blocking mode. A better mode to save power during timeout will be
> > to use umwait-based sleep.  Consider this as extension to current timeout
> behavior if device supports interrupts.
> >
> 
> What about applications that both want power efficiency *and* require a
> timeout?
> 
> It makes no sense to me the change the semantics of the dequeue timeout
> parameter from "the time until I want control back" to "the time I want the
> event device to use polling, after which it should put the thread to sleep".
> Those two are pretty much orthogonal.
> 
> The current API doesn't specify what happens during the timeout. If you by
> "non-blocking" mean "busy-polling", that is not an API requirement. I don't
> see why the event device couldn't put the lcore thread to sleep during a long
> timeout (and I also see why you may not want that to be the default
> behavior).

I got your point. Application will prefer to return and not block after the timeout as 
per current API description. Current API signature is 
= 0 no-wait, returns immediately if there is no event.
>0 wait for the event or timeout
Will it be OK to extend it to treat max timeout 0xffffffff as block for event with PMD specific 
Mechanism to wait for the event ? Port specific configuration will not be required in this case.
Devices supporting interrupts then can use interrupt blocking mode internally. 

> 
> >>
> >>> If not interrupt, it can be called blocking vs polling mode.  If the
> >>> port config is
> >> fine, I will create a new patch with it.
> >>> Also, we should have this as a capability for eventdevs.
> >>>
> >>>>
> >>>>>
> >>>>> What's being configured should just be a threshold time at which
> >>>>> the event device would go from busy-polling to blocking the thread.
> >>>>>
> >>>>> Maybe it should be called something with "blocking" or "sleeping",
> >>>>> instead of "interrupt", since interrupts are never directly involved.
> >>>>
> >>>> Agree. or make it a power save mode or so.
> >>>>
> >>>>>
> >>>>> Anyway, seems like a good candidate for a generic feature to me.
> >>>>>
> >