[EXT] Re: [RFC 0/3] Introduce event link profiles

[Mattias Rönnblom]

On 2023-08-10 07:17, Pavan Nikhilesh Bhagavatula wrote:
>> On 2023-08-09 16:26, pbhagavatula at marvell.com wrote:
>>> From: Pavan Nikhilesh <pbhagavatula at marvell.com>
>>>
>>> A collection of event queues linked to an event port can be associated
>>> with unique identifier called as a profile, multiple such profiles can
>>> be configured based on the event device capability using the function
>>> `rte_event_port_link_with_profile` which takes arguments similar to
>>> `rte_event_port_link` in addition to the profile identifier.
>>>
>>
>> What is the overall goal with this new API? What problems does it intend
>> to solve, that the old one doesn't.
> 
> Linking and unlinking currently has huge overhead and when it needs to be done
> in fastpath, we have to wait for unlinks to complete and handle other corner cases.
> 

OK, so this API change is specific to some particular hardware? Is this 
true for some other event devices? That "huge overhead" goes to "simple 
function call" for unlinking+linking, provided the target configuration 
is known in advance.

What is the overall use case?

> This patch set solves it by avoiding linking/unlinking altogether in fastpath by
> preconfigured set of link profiles out of which only one would be active and can
> be changed in fastpath with a simple function call. There is no link/unlink waiting for
> unlink overhead.
> 
>>
>>> The maximum link profiles that are supported by an event device is
>>> advertised through the structure member
>>> `rte_event_dev_info::max_profiles_per_port`.
>>>
>>> By default, event ports are configured to use the link profile 0 on
>>> initialization.
>>>
>>> Once multiple link profiles are set up and the event device is started, the
>>> application can use the function `rte_event_port_change_profile` to
>> change
>>> the currently active profile on an event port. This effects the next
>>> `rte_event_dequeue_burst` call, where the event queues associated with
>> the
>>> newly active link profile will participate in scheduling.
>>>
>>> Rudementary work flow would something like:
>>>
>>> Config path:
>>>
>>>       uint8_t lowQ[4] = {4, 5, 6, 7};
>>>       uint8_t highQ[4] = {0, 1, 2, 3};
>>>
>>>       if (rte_event_dev_info.max_profiles_per_port < 2)
>>>           return -ENOTSUP;
>>>
>>>       rte_event_port_link_with_profile(0, 0, highQ, NULL, 4, 0);
>>>       rte_event_port_link_with_profile(0, 0, lowQ, NULL, 4, 1);
>>>
>>> Worker path:
>>>
>>>       empty_high_deq = 0;
>>>       empty_low_deq = 0;
>>>       is_low_deq = 0;
>>>       while (1) {
>>>           deq = rte_event_dequeue_burst(0, 0, &ev, 1, 0);
>>>           if (deq == 0) {
>>>               /**
>>>                * Change link profile based on work activity on current
>>>                * active profile
>>>                */
>>>               if (is_low_deq) {
>>>                   empty_low_deq++;
>>>                   if (empty_low_deq == MAX_LOW_RETRY) {
>>>                       rte_event_port_change_profile(0, 0, 0);
>>>                       is_low_deq = 0;
>>>                       empty_low_deq = 0;
>>>                   }
>>>                   continue;
>>>               }
>>>
>>>               if (empty_high_deq == MAX_HIGH_RETRY) {
>>>                   rte_event_port_change_profile(0, 0, 1);
>>>                   is_low_deq = 1;
>>>                   empty_high_deq = 0;
>>>               }
>>>               continue;
>>>           }
>>>
>>>           // Process the event received.
>>>
>>>           if (is_low_deq++ == MAX_LOW_EVENTS) {
>>>               rte_event_port_change_profile(0, 0, 0);
>>>               is_low_deq = 0;
>>>           }
>>>       }
>>>
>>
>> This thing looks like the application is asked to do work scheduling.
>> That doesn't sound right. That's the job of the work scheduler (i.e.,
>> the event device).
>>
>> If this thing is merely a matter of changing what queues are linked to
>> which ports, wouldn't a new call:
>> rte_event_port_link_modify()
>> suffice?
> 
> 
> Some applications divide their available lcores into multiple types of
> workers which each work on a unique set of event queues, application might
> need to modify the worker ratio based on various parameters at run time
> without a lot of overhead.
> 
> Modifying links wouldn’t work because we might want to restore previous links
> based on the new traffic pattern etc.,.
> 
>>
>>> An application could use heuristic data of load/activity of a given event
>>> port and change its active profile to adapt to the traffic pattern.
>>>
>>> An unlink function `rte_event_port_unlink_with_profile` is provided to
>>> modify the links associated to a profile, and
>>> `rte_event_port_links_get_with_profile` can be used to retrieve the links
>>> associated with a profile.
>>>
>>> Pavan Nikhilesh (3):
>>>     eventdev: introduce link profiles
>>>     event/cnxk: implement event link profiles
>>>     test/event: add event link profile test
>>>
>>>    app/test/test_eventdev.c                   | 110 ++++++++++
>>>    config/rte_config.h                        |   1 +
>>>    doc/guides/eventdevs/cnxk.rst              |   1 +
>>>    doc/guides/prog_guide/eventdev.rst         |  58 ++++++
>>>    drivers/common/cnxk/roc_nix_inl_dev.c      |   4 +-
>>>    drivers/common/cnxk/roc_sso.c              |  18 +-
>>>    drivers/common/cnxk/roc_sso.h              |   8 +-
>>>    drivers/common/cnxk/roc_sso_priv.h         |   4 +-
>>>    drivers/event/cnxk/cn10k_eventdev.c        |  45 ++--
>>>    drivers/event/cnxk/cn10k_worker.c          |  11 +
>>>    drivers/event/cnxk/cn10k_worker.h          |   1 +
>>>    drivers/event/cnxk/cn9k_eventdev.c         |  72 ++++---
>>>    drivers/event/cnxk/cn9k_worker.c           |  22 ++
>>>    drivers/event/cnxk/cn9k_worker.h           |   2 +
>>>    drivers/event/cnxk/cnxk_eventdev.c         |  34 ++--
>>>    drivers/event/cnxk/cnxk_eventdev.h         |  10 +-
>>>    drivers/event/dlb2/dlb2.c                  |   1 +
>>>    drivers/event/dpaa/dpaa_eventdev.c         |   1 +
>>>    drivers/event/dpaa2/dpaa2_eventdev.c       |   2 +-
>>>    drivers/event/dsw/dsw_evdev.c              |   1 +
>>>    drivers/event/octeontx/ssovf_evdev.c       |   2 +-
>>>    drivers/event/opdl/opdl_evdev.c            |   1 +
>>>    drivers/event/skeleton/skeleton_eventdev.c |   1 +
>>>    drivers/event/sw/sw_evdev.c                |   1 +
>>>    lib/eventdev/eventdev_pmd.h                |  59 +++++-
>>>    lib/eventdev/eventdev_private.c            |   9 +
>>>    lib/eventdev/eventdev_trace.h              |  22 ++
>>>    lib/eventdev/eventdev_trace_points.c       |   6 +
>>>    lib/eventdev/rte_eventdev.c                | 146 ++++++++++---
>>>    lib/eventdev/rte_eventdev.h                | 226 +++++++++++++++++++++
>>>    lib/eventdev/rte_eventdev_core.h           |   4 +
>>>    lib/eventdev/rte_eventdev_trace_fp.h       |   8 +
>>>    lib/eventdev/version.map                   |   5 +
>>>    33 files changed, 788 insertions(+), 108 deletions(-)
>>>
>>> --
>>> 2.25.1
>>>