[RFC v3 2/2] ethdev: introduce the cache stashing hints API

[Wathsala Wathawana Vithanage]

> -----Original Message-----
> From: Jerin Jacob <jerinjacobk at gmail.com>
> Sent: Thursday, October 24, 2024 12:49 AM
> To: Wathsala Wathawana Vithanage <wathsala.vithanage at arm.com>
> Cc: thomas at monjalon.net; Ferruh Yigit <ferruh.yigit at amd.com>; Andrew
> Rybchenko <andrew.rybchenko at oktetlabs.ru>; dev at dpdk.org; nd
> <nd at arm.com>; Honnappa Nagarahalli <Honnappa.Nagarahalli at arm.com>;
> Dhruv Tripathi <Dhruv.Tripathi at arm.com>
> Subject: Re: [RFC v3 2/2] ethdev: introduce the cache stashing hints API
> 
> On Mon, Oct 21, 2024 at 7:23 AM Wathsala Vithanage
> <wathsala.vithanage at arm.com> wrote:
> >
> > Extend the ethdev library to enable the stashing of different data
> > objects, such as the ones listed below, into CPU caches directly from
> > the NIC.
> >
> > - Rx/Tx queue descriptors
> > - Rx packets
> > - Packet headers
> > - packet payloads
> > - Data of a packet at an offset from the start of the packet
> >
> > The APIs are designed in a hardware/vendor agnostic manner such that
> > supporting PMDs could use any capabilities available in the underlying
> > hardware for fine-grained stashing of data objects into a CPU cache
> > (e.g., Steering Tags int PCIe TLP Processing Hints).
> >
> > The API provides an interface to query the availability of stashing
> > capabilities, i.e., platform/NIC support, stashable object types, etc,
> > via the rte_eth_dev_stashing_capabilities_get interface.
> >
> > The function pair rte_eth_dev_stashing_rx_config_set and
> > rte_eth_dev_stashing_tx_config_set sets the stashing hint (the CPU,
> > cache level, and data object types) on the Rx and Tx queues.
> >
> > PMDs that support stashing must register their implementations with
> > the following eth_dev_ops callbacks, which are invoked by the ethdev
> > functions listed above.
> >
> > - stashing_capabilities_get
> > - stashing_rx_hints_set
> > - stashing_tx_hints_set
> >
> > Signed-off-by: Wathsala Vithanage <wathsala.vithanage at arm.com>
> > Reviewed-by: Honnappa Nagarahalli <honnappa.nagarahalli at arm.com>
> > Reviewed-by: Dhruv Tripathi <dhruv.tripathi at arm.com>
> >
> 
> > +
> > +/** Queue type is RX. */
> > +#define RTE_ETH_DEV_RX_QUEUE           0
> > +/** Queue type is TX. */
> > +#define RTE_ETH_DEV_TX_QUEUE           1
> > +
> > +
> > +/**
> > + * @warning
> > + * @b EXPERIMENTAL: this structure may change, or be removed, without
> > +prior notice
> > + *
> > + * A structure used for configuring the cache stashing hints.
> > + */
> > +struct rte_eth_stashing_config {
> > +       /** ID of the Processor/Container the stashing hints are
> > +        *  applied to
> > +        */
> > +       uint16_t        lcore_id;
> > +       /** Set if the target is a CPU containeri.lcore_id will be
> > +        * used to derive container ID
> > +        */
> > +       uint16_t        container : 1;
> > +       uint16_t        padding : 7;
> > +       /** Cache level of the CPU specified by the cpu_id the
> > +        *  stashing hints are applied to
> > +        */
> > +       uint16_t        cache_level : 8;
> > +       /** Object types the configuration is applied to
> > +        */
> > +       uint16_t        objects;
> > +       /** The offset if RTE_ETH_DEV_STASH_OBJECT_OFFSET bit is set
> > +        *  in objects
> > +        */
> > +       off_t           offset;
> > +};
> > +
> > +/**@{@name Stashable Rx/Tx queue object types supported by the
> > +ethernet device  *@see rte_eth_dev_stashing_capabilities_get
> > + *@see rte_eth_dev_stashing_rx_config_set
> > + *@see rte_eth_dev_stashing_tx_config_set
> > + */
> > +
> > +/**
> > + * Apply stashing hint to data at a given offset from the start of a
> > + * received packet.
> > + */
> > +#define RTE_ETH_DEV_STASH_OBJECT_OFFSET                0x0001
> > +
> > +/** Apply stashing hint to an rx descriptor. */
> > +#define RTE_ETH_DEV_STASH_OBJECT_DESC          0x0002
> > +
> > +/** Apply stashing hint to a header of a received packet. */
> > +#define RTE_ETH_DEV_STASH_OBJECT_HEADER                0x0004
> > +
> > +/** Apply stashing hint to a payload of a received packet. */
> > +#define RTE_ETH_DEV_STASH_OBJECT_PAYLOAD       0x0008
> > +
> > +#define __RTE_ETH_DEV_STASH_OBJECT_MASK                0x000f
> > +/**@}*/
> > +
> > +#define RTE_ETH_DEV_STASH_OBJECTS_VALID(t)                             \
> > +       ((!((t) & (~__RTE_ETH_DEV_STASH_OBJECT_MASK))) && (t))
> > +
> 
> 
> I think, at one of point of time, we need to extend this to other device class
> like(cryptodev etc) where the data needs to move over bus. In that context, all
> the above symbols better to be in EAL and the device class subsystem(example
> ethdev) gives PMD callback.

+1
I will make that change in the RFC v4.