[RFC 0/6] net/mlx5: introduce limit watermark and host shaper

[Jerin Jacob]

On Tue, Apr 26, 2022 at 8:12 AM Spike Du <spiked at nvidia.com> wrote:
>
> Hi Jerin,

Hi Spike,

>         Thanks for your comments and sorry for the late response.
>
>         For case one, I think I can refine the design and add LWM(limit watermark) in rte_eth_rxconf, and add a new rte_eth_event_type event.

OK.

>
>         For case two(host shaper), I think we can't use RX meter, because it's actually TX shaper on a remote system. It's quite specific to Mellanox/Nvidia BlueField 2(BF2 for short) NIC. The NIC contains an ARM system. We have two terms here: Host-system stands for the system the BF2 NIC is inserted; ARM-system stands for the embedded ARM in BF2. ARM-system is doing the forwarding. This is the way host shaper works: we configure the register on ARM-system, but it affects Host-system's TX shaper, which means the shaper is working on the remote port, it's not a RX meter concept, hence we can't use DPDK RX meter framework. I'd suggest to still use private API.

OK. If the host is using the DPDK application then rte_tm can be used
on the egress side to enable the same. If it is not DPDK, then yes, we
need private APIs.

>
>         For testpmd part, I understand your concern. Because we need one private API for host shaper, and we need testpmd's forwarding code to show how it works to user, we need to call the private API in testpmd. If current patch is not acceptable, what's the correct way to do it? Any framework to isolate the PMD private logic from testpmd common code, but still give a chance to call private APIs in testpmd?

Please check "PMD API" item in
http://mails.dpdk.org/archives/dev/2022-April/239191.html

>
>
> Regards,
> Spike.
>
>
>
> > -----Original Message-----
> > From: Jerin Jacob <jerinjacobk at gmail.com>
> > Sent: Tuesday, April 5, 2022 4:59 PM
> > To: Spike Du <spiked at nvidia.com>; Andrew Rybchenko
> > <andrew.rybchenko at oktetlabs.ru>; Cristian Dumitrescu
> > <cristian.dumitrescu at intel.com>; Ferruh Yigit <ferruh.yigit at intel.com>;
> > techboard at dpdk.org
> > Cc: Matan Azrad <matan at nvidia.com>; Slava Ovsiienko
> > <viacheslavo at nvidia.com>; Ori Kam <orika at nvidia.com>; NBU-Contact-
> > Thomas Monjalon (EXTERNAL) <thomas at monjalon.net>; dpdk-dev
> > <dev at dpdk.org>; Raslan Darawsheh <rasland at nvidia.com>
> > Subject: Re: [RFC 0/6] net/mlx5: introduce limit watermark and host shaper
> >
> > External email: Use caution opening links or attachments
> >
> >
> > On Fri, Apr 1, 2022 at 8:53 AM Spike Du <spiked at nvidia.com> wrote:
> > >
> > > LWM(limit watermark) is per RX queue attribute, when RX queue fullness
> > > reach the LWM limit, HW sends an event to dpdk application.
> > > Host shaper can configure shaper rate and lwm-triggered for a host port.
> > > The shaper limits the rate of traffic from host port to wire port.
> > > If lwm-triggered is enabled, a 100Mbps shaper is enabled automatically
> > > when one of the host port's Rx queues receives LWM event.
> > >
> > > These two features can combine to control traffic from host port to wire
> > port.
> > > The work flow is configure LWM to RX queue and enable lwm-triggered
> > > flag in host shaper, after receiving LWM event, delay a while until RX
> > > queue is empty , then disable the shaper. We recycle this work flow to
> > reduce RX queue drops.
> > >
> > > Spike Du (6):
> > >   net/mlx5: add LWM support for Rxq
> > >   common/mlx5: share interrupt management
> > >   net/mlx5: add LWM event handling support
> > >   net/mlx5: add private API to configure Rxq LWM
> > >   net/mlx5: add private API to config host port shaper
> > >   app/testpmd: add LWM and Host Shaper command
> >
> > + @Andrew Rybchenko  @Ferruh Yigit cristian.dumitrescu at intel.com
> >
> > I think, case one, can be easily abstracted via adding new
> > rte_eth_event_type event and case two can be abstracted via the existing
> > Rx meter framework in ethdev.
> >
> > Also, Updating generic testpmd to support PMD specific API should be
> > avoided, I know there is existing stuff in testpmd, I think, we should have the
> > policy to add PMD specific commands to testpmd.
> >
> > There are around 56PMDs in ethdev now, If PMDs try to add PMD specific
> > API in testpmd it will be bloated or at minimum, it should a separate file in
> > testpmd if we choose to take that path.
> >
> > + @techboard at dpdk.org