[dpdk-dev] [RFC] ethdev: support metadata as flow rule criteria

[Ananyev, Konstantin]

Hi Dekel,

> >
> >
> > >
> > > > -----Original Message-----
> > > > From: Dekel Peled [mailto:dekelp at mellanox.com]
> > > > Sent: Monday, August 13, 2018 10:47 AM
> > > > To: dev at dpdk.org
> > > > Cc: Ori Kam <orika at mellanox.com>; Shahaf Shuler
> > > > <shahafs at mellanox.com>
> > > > Subject: [RFC] ethdev: support metadata as flow rule criteria
> > > >
> > > > Current implementation of rte_flow allows match pattern of flow
> > > > rule, based on packet data or header fields.
> > > > This limits the application use of match patterns.
> > > >
> > > > For example, consider a vswitch application which controls a set of
> > > > VMs, connected with virtio, in a fabric with overlay of VXLAN.
> > > > Several VMs can have the same inner tuple, while the outer tuple is
> > > > different and controlled by the vswitch (encap action).
> > > > For the vswtich to be able to offload the rule to the NIC, it must
> > > > use a unique match criteria, independent from the inner tuple, to
> > > > perform the encap action.
> > > >
> > > > This RFC adds support for additional metadata to use as match pattern.
> > > > The metadata is an opaque item, fully controlled by the application.
> > > >
> > > > The use of metadata is relevant for egress rules only.
> > > > It can be set in the flow rule using the RTE_FLOW_ITEM_META.
> > > >
> > > > Application should set the packet metdata in the mbuf->metadata
> > > > field, and set the PKT_TX_METADATA flag in the mbuf->ol_flags.
> > > > The NIC will use the packet metadata as match criteria for relevant flow
> > rules.
> > > >
> > > > For example, to do an encap action depending on the VM id, the
> > > > application needs to configure 'match on metadata' rte_flow rule
> > > > with VM id as metadata, along with desired encap action.
> > > > When preparing an egress data packet, application will set VM id
> > > > data in mbuf metadata field and set PKT_TX_METADATA flag.
> > > >
> > > > PMD will send data packets to NIC, with VM id as metadata.
> > > > Egress flow on NIC will match metadata as done with other criteria.
> > > > Upon match on metadata (VM id) the appropriate encap action will be
> > > > performed.
> > > >
> > > > This RFC introduces metadata item type for rte_flow
> > > > RTE_FLOW_ITEM_META, along with corresponding struct
> > > > rte_flow_item_meta and ol_flag PKT_TX_METADATA.
> > > > It also enhances struct rte_mbuf with new data item, uint64_t metadata.
> > > >
> > > > Comments are welcome.
> > > >
> > > > Signed-off-by: Dekel Peled <dekelp at mellanox.com>
> > > > ---
> > > >  doc/guides/prog_guide/rte_flow.rst | 21 +++++++++++++++++++++
> > > >  lib/librte_ethdev/rte_flow.c       |  1 +
> > > >  lib/librte_ethdev/rte_flow.h       | 25 +++++++++++++++++++++++++
> > > >  lib/librte_mbuf/rte_mbuf.h         | 11 +++++++++++
> > > >  4 files changed, 58 insertions(+)
> > > >
> > > > diff --git a/doc/guides/prog_guide/rte_flow.rst
> > > > b/doc/guides/prog_guide/rte_flow.rst
> > > > index b305a72..b6e35f1 100644
> > > > --- a/doc/guides/prog_guide/rte_flow.rst
> > > > +++ b/doc/guides/prog_guide/rte_flow.rst
> > > > @@ -1191,6 +1191,27 @@ Normally preceded by any of:
> > > >  - `Item: ICMP6_ND_NS`_
> > > >  - `Item: ICMP6_ND_OPT`_
> > > >
> > > > +Item: ``META``
> > > > +^^^^^^^^^^^^^^
> > > > +
> > > > +Matches an application specific 64 bit metadata item.
> > > > +
> > > > +- Default ``mask`` matches any 64 bit value.
> > > > +
> > > > +.. _table_rte_flow_item_meta:
> > > > +
> > > > +.. table:: META
> > > > +
> > > > +   +----------+----------+---------------------------+
> > > > +   | Field    | Subfield | Value                     |
> > > > +   +==========+==========+===========================+
> > > > +   | ``spec`` | ``data`` | 64 bit metadata value     |
> > > > +   +----------+--------------------------------------+
> > > > +   | ``last`` | ``data`` | upper range value         |
> > > > +   +----------+----------+---------------------------+
> > > > +   | ``mask`` | ``data`` | zeroed to match any value |
> > > > +   +----------+----------+---------------------------+
> > > > +
> > > >  Actions
> > > >  ~~~~~~~
> > > >
> > > > diff --git a/lib/librte_ethdev/rte_flow.c
> > > > b/lib/librte_ethdev/rte_flow.c index
> > > > cff4b52..54e5ef8 100644
> > > > --- a/lib/librte_ethdev/rte_flow.c
> > > > +++ b/lib/librte_ethdev/rte_flow.c
> > > > @@ -66,6 +66,7 @@ struct rte_flow_desc_data {
> > > >  		     sizeof(struct rte_flow_item_icmp6_nd_opt_sla_eth)),
> > > >  	MK_FLOW_ITEM(ICMP6_ND_OPT_TLA_ETH,
> > > >  		     sizeof(struct rte_flow_item_icmp6_nd_opt_tla_eth)),
> > > > +	MK_FLOW_ITEM(META, sizeof(struct rte_flow_item_meta)),
> > > >  };
> > > >
> > > >  /** Generate flow_action[] entry. */ diff --git
> > > > a/lib/librte_ethdev/rte_flow.h b/lib/librte_ethdev/rte_flow.h index
> > > > f8ba71c..b81c816 100644
> > > > --- a/lib/librte_ethdev/rte_flow.h
> > > > +++ b/lib/librte_ethdev/rte_flow.h
> > > > @@ -413,6 +413,15 @@ enum rte_flow_item_type {
> > > >  	 * See struct rte_flow_item_mark.
> > > >  	 */
> > > >  	RTE_FLOW_ITEM_TYPE_MARK,
> > > > +
> > > > +	/**
> > > > +	 * [META]
> > > > +	 *
> > > > +	 * Matches a metadata value specified in mbuf metadata field.
> > > > +	 *
> > > > +	 * See struct rte_flow_item_meta.
> > > > +	 */
> > > > +	RTE_FLOW_ITEM_TYPE_META,
> > > >  };
> > > >
> > > >  /**
> > > > @@ -849,6 +858,22 @@ struct rte_flow_item_gre {  #endif
> > > >
> > > >  /**
> > > > + * RTE_FLOW_ITEM_TYPE_META.
> > > > + *
> > > > + * Matches a specified metadata value.
> > > > + */
> > > > +struct rte_flow_item_meta {
> > > > +	uint64_t data;
> > > > +};
> > > > +
> > > > +/** Default mask for RTE_FLOW_ITEM_TYPE_META. */ #ifndef
> > > > +__cplusplus static const struct rte_flow_item_meta
> > rte_flow_item_meta_mask = {
> > > > +	.data = RTE_BE64(UINT64_MAX),
> > > > +};
> > > > +#endif
> > > > +
> > > > +/**
> > > >   * RTE_FLOW_ITEM_TYPE_FUZZY
> > > >   *
> > > >   * Fuzzy pattern match, expect faster than default.
> > > > diff --git a/lib/librte_mbuf/rte_mbuf.h b/lib/librte_mbuf/rte_mbuf.h
> > > > index
> > > > 9ce5d76..8f06a78 100644
> > > > --- a/lib/librte_mbuf/rte_mbuf.h
> > > > +++ b/lib/librte_mbuf/rte_mbuf.h
> > > > @@ -182,6 +182,11 @@
> > > >  /* add new TX flags here */
> > > >
> > > >  /**
> > > > + * This flag indicates that the metadata field in the mbuf is in use.
> > > > + */
> > > > +#define PKT_TX_METADATA		(1ULL << 41)
> > > > +
> > > > +/**
> > > >   * UDP Fragmentation Offload flag. This flag is used for enabling UDP
> > > >   * fragmentation in SW or in HW. When use UFO, mbuf->tso_segsz is
> > used
> > > >   * to store the MSS of UDP fragments.
> > > > @@ -593,6 +598,12 @@ struct rte_mbuf {
> > > >  	 */
> > > >  	struct rte_mbuf_ext_shared_info *shinfo;
> > > >
> > > > +	/**
> > > > +	 * Application specific metadata value for flow rule match.
> > > > +	 * Valid if PKT_TX_METADATA is set.
> > > > +	 */
> > > > +	uint64_t metadata;
> > > > +
> >
> > Just one thought - with that change we'll have only 8 free bytes left inside
> > rte_mbuf.
> > Wonder tan this metadata field be combined within tx_offload or probably
> > hash fields?
> > Konstantin
> 
> The match on metadata feature is currently implemented for egress, but is planned to be extended for ingress use in the future.
> Hence the need for dedicated field, detached from Tx specific or Rx specific fields.

Could you probably explain a bit more how it will be used for ingress?
As I understand it would be some user defined value associated with particular HW filter.
Right now mbuf's hash might be used for similar purposes - it can contain flow filter ID.
Do you expect HW to provide both rss/flow and this new metadata info simultaneously
for the same packet?
Konstantin

> Dekel
> 
> >
> >
> > > >  } __rte_cache_aligned;
> > > >
> > > >  /**
> > > > --
> > > > 1.8.3.1