[dpdk-dev] [RFC v2] ethdev: support metadata as flow rule criteria

[Dekel Peled]

> -----Original Message-----
> From: Yongseok Koh
> Sent: Tuesday, August 28, 2018 10:44 PM
> To: Dekel Peled <dekelp at mellanox.com>
> Cc: dev <dev at dpdk.org>; Shahaf Shuler <shahafs at mellanox.com>; Ori Kam
> <orika at mellanox.com>; Andrew Rybchenko <arybchenko at solarflare.com>;
> Yigit, Ferruh <ferruh.yigit at intel.com>; Thomas Monjalon
> <thomas at monjalon.net>; Ananyev, Konstantin
> <konstantin.ananyev at intel.com>; Adrien Mazarguil
> <adrien.mazarguil at 6wind.com>; Olivier Matz <olivier.matz at 6wind.com>;
> Alex Rosenbaum <alexr at mellanox.com>
> Subject: Re: [RFC v2] ethdev: support metadata as flow rule criteria
> 
> > On Aug 26, 2018, at 7:09 AM, Dekel Peled <dekelp at mellanox.com> wrote:
> >
> > Current implementation of rte_flow allows match pattern of flow rule,
> > based on packet data or header fields.
> > This limits the application use of match patterns.
> >
> > For example, consider a vswitch application which controls a set of
> > VMs, connected with virtio, in a fabric with overlay of VXLAN.
> > Several VMs can have the same inner tuple, while the outer tuple is
> > different and controlled by the vswitch (encap action).
> > For the vswtich to be able to offload the rule to the NIC, it must use
> > a unique match criteria, independent from the inner tuple, to perform
> > the encap action.
> >
> > This RFC adds support for additional metadata to use as match pattern.
> > The metadata is an opaque item, fully controlled by the application.
> >
> > The use of metadata is relevant for egress rules only.
> > It can be set in the flow rule using the RTE_FLOW_ITEM_META.
> >
> > In order to avoid change in mbuf API, exisitng field mbuf.hash.fdir.hi
> > will be used to carry the metadata item. This field is used only in
> > ingress packets, so using it for egress metadata will not cause conflicts.
> >
> > Application should set the packet metdata in the mbuf dedicated field,
> > and set the PKT_TX_METADATA flag in the mbuf->ol_flags.
> > The NIC will use the packet metadata as match criteria for relevant
> > flow rules.
> >
> > For example, to do an encap action depending on the VM id, the
> > application needs to configure 'match on metadata' rte_flow rule with
> > VM id as metadata, along with desired encap action.
> > When preparing an egress data packet, application will set VM id data
> > in mbuf dedicated field, and set PKT_TX_METADATA flag.
> >
> > PMD will send data packets to NIC, with VM id as metadata.
> > Egress flow on NIC will match metadata as done with other criteria.
> > Upon match on metadata (VM id) the appropriate encap action will be
> > performed.
> >
> > This RFC introduces metadata item type for rte_flow
> > RTE_FLOW_ITEM_META, along with corresponding struct
> rte_flow_item_meta
> > and ol_flag PKT_TX_METADATA.
> >
> > Comments are welcome.
> >
> > Signed-off-by: Dekel Peled <dekelp at mellanox.com>
> > ---
> > v2: Use existing field in mbuf for metadata item, as suggested, instead
> >    of adding a new field.
> >    Metadata item size adjusted to 32 bits.
> > ---
> > doc/guides/prog_guide/rte_flow.rst | 21 +++++++++++++++++++++
> > lib/librte_ethdev/rte_flow.c       |  1 +
> > lib/librte_ethdev/rte_flow.h       | 25 +++++++++++++++++++++++++
> > lib/librte_mbuf/rte_mbuf.h         | 13 +++++++++++++
> > 4 files changed, 60 insertions(+)
> >
> > diff --git a/doc/guides/prog_guide/rte_flow.rst
> > b/doc/guides/prog_guide/rte_flow.rst
> > index b305a72..560e45a 100644
> > --- a/doc/guides/prog_guide/rte_flow.rst
> > +++ b/doc/guides/prog_guide/rte_flow.rst
> > @@ -1191,6 +1191,27 @@ Normally preceded by any of:
> > - `Item: ICMP6_ND_NS`_
> > - `Item: ICMP6_ND_OPT`_
> >
> > +Item: ``META``
> > +^^^^^^^^^^^^^^
> > +
> > +Matches an application specific 32 bit metadata item.
> > +
> > +- Default ``mask`` matches any 32 bit value.
> > +
> > +.. _table_rte_flow_item_meta:
> > +
> > +.. table:: META
> > +
> > +   +----------+----------+---------------------------+
> > +   | Field    | Subfield | Value                     |
> > +   +==========+==========+===========================+
> > +   | ``spec`` | ``data`` | 32 bit metadata value     |
> > +   +----------+--------------------------------------+
> > +   | ``last`` | ``data`` | upper range value         |
> > +   +----------+----------+---------------------------+
> > +   | ``mask`` | ``data`` | zeroed to match any value |
> > +   +----------+----------+---------------------------+
> > +
> > Actions
> > ~~~~~~~
> >
> > diff --git a/lib/librte_ethdev/rte_flow.c
> > b/lib/librte_ethdev/rte_flow.c index cff4b52..54e5ef8 100644
> > --- a/lib/librte_ethdev/rte_flow.c
> > +++ b/lib/librte_ethdev/rte_flow.c
> > @@ -66,6 +66,7 @@ struct rte_flow_desc_data {
> > 		     sizeof(struct rte_flow_item_icmp6_nd_opt_sla_eth)),
> > 	MK_FLOW_ITEM(ICMP6_ND_OPT_TLA_ETH,
> > 		     sizeof(struct rte_flow_item_icmp6_nd_opt_tla_eth)),
> > +	MK_FLOW_ITEM(META, sizeof(struct rte_flow_item_meta)),
> > };
> >
> > /** Generate flow_action[] entry. */
> > diff --git a/lib/librte_ethdev/rte_flow.h
> > b/lib/librte_ethdev/rte_flow.h index f8ba71c..eba3cc4 100644
> > --- a/lib/librte_ethdev/rte_flow.h
> > +++ b/lib/librte_ethdev/rte_flow.h
> > @@ -413,6 +413,15 @@ enum rte_flow_item_type {
> > 	 * See struct rte_flow_item_mark.
> > 	 */
> > 	RTE_FLOW_ITEM_TYPE_MARK,
> > +
> > +	/**
> > +	 * [META]
> > +	 *
> > +	 * Matches a metadata value specified in mbuf metadata field.
> > +	 *
> > +	 * See struct rte_flow_item_meta.
> > +	 */
> > +	RTE_FLOW_ITEM_TYPE_META,
> > };
> >
> > /**
> > @@ -849,6 +858,22 @@ struct rte_flow_item_gre { #endif
> >
> > /**
> > + * RTE_FLOW_ITEM_TYPE_META.
> > + *
> > + * Matches a specified metadata value.
> > + */
> > +struct rte_flow_item_meta {
> > +	uint32_t data;
> > +};
> > +
> > +/** Default mask for RTE_FLOW_ITEM_TYPE_META. */ #ifndef
> __cplusplus
> > +static const struct rte_flow_item_meta rte_flow_item_meta_mask = {
> > +	.data = RTE_BE32(UINT32_MAX),
> > +};
> > +#endif
> > +
> > +/**
> >  * RTE_FLOW_ITEM_TYPE_FUZZY
> >  *
> >  * Fuzzy pattern match, expect faster than default.
> > diff --git a/lib/librte_mbuf/rte_mbuf.h b/lib/librte_mbuf/rte_mbuf.h
> > index 9ce5d76..77c1552 100644
> > --- a/lib/librte_mbuf/rte_mbuf.h
> > +++ b/lib/librte_mbuf/rte_mbuf.h
> > @@ -182,6 +182,11 @@
> > /* add new TX flags here */
> >
> > /**
> > + * This flag indicates that the metadata field in the mbuf is in use.
> > + */
> > +#define PKT_TX_METADATA		(1ULL << 41)
> > +
> > +/**
> >  * UDP Fragmentation Offload flag. This flag is used for enabling UDP
> >  * fragmentation in SW or in HW. When use UFO, mbuf->tso_segsz is used
> >  * to store the MSS of UDP fragments.
> > @@ -526,6 +531,14 @@ struct rte_mbuf {
> > 			uint32_t hi;
> > 			/**< First 4 flexible bytes or FD ID, dependent on
> > 			     PKT_RX_FDIR_* flag in ol_flags. */
> > +
> > +			/**
> > +			 * Above item has optional use on egress:
> > +			 * Application specific metadata value
> > +			 * for flow rule match.
> > +			 * Valid if PKT_TX_METADATA is set.
> > +			 */
> > +
> 
> Hi Dekel,
> 
> I don't think we have reached to a conclusion?? I remember there were
> three options.
> 1) add a new 64bit field
> 2) use userdata/udata64
> 3) use hash
> 
> I still prefer 1) but if people here think that more fields will have to be added
> in the near feature then 2) would be my next preference. But, if we just
> have some unclear anxiety (like the depletion of IPv4 address :-), 1) would
> still be good.
> 
> But, 3) is my least preference as a Rx mbuf still can have both flow ID and
> metadata.
> 
> We still need more input/discussion.

Option 1 was not favored in discussions so far, see RFC email chain.
Option 2 is unwanted since there may be applications using userdata/udata64.
Currently we see use of metadata in tx only, hence option 3 is preferred.

> 
> Thanks,
> Yongseok
> 
> > 		} fdir;           /**< Filter identifier if FDIR enabled */
> > 		struct {
> > 			uint32_t lo;
> > --
> > 1.8.3.1
> >