[dpdk-dev] [RFC] ethdev: flow counters batch query

[Slava Ovsiienko]

Adding relevant maintainers.

> -----Original Message-----
> From: Slava Ovsiienko
> Sent: Monday, August 20, 2018 14:39
> To: dev at dpdk.org
> Cc: Shahaf Shuler <shahafs at mellanox.com>; Slava Ovsiienko
> <viacheslavo at mellanox.com>
> Subject: [RFC] ethdev: flow counters batch query
> 
> There is a demand to perform query operations on the set of counters
> generally belonging to the different RTE flows. The counter queries is the
> very effective way for application to know in detail what is going on into the
> network and to take some actions basing on this knowledge. As example, the
> modern vSwitch implementation supporting full hardware offload may need
> a huge number of counters and query these ones periodically for
> configuration aging check purposes.
> 
> For some devices the counter query can be very expensive operation and
> can take a significant amount of time - often it involves the calls of kernel
> mode components and performing requests to hardware.
> If application wants to query multiple counters, belonging to the different
> RTE flows, it has to execute queries in one-by-one fashion due to the current
> RTE API implementation that allows to query counters belonging to the same
> flow only. The proposed RTE API extension introduces the compatible and
> more performance way to allow applications to query the counter values as a
> batch.
> 
> At the creation the counter can be optionally assigned with the new
> introduced 'group_id' identifier (batch identifier). All counters with the same
> group_id form the new introduced entity
> - 'counter batch'. The group_id specifies the unique batch within device with
> given port_id. The same group_id may specify the different batches on
> different devices.
> 
> The new method 'rte_flow_query_update()' is proposed to perform the
> counter batch query from the device as a single call, this eliminates a much of
> overhead involved with multiple quieries for the counters belonging to the
> different flows. The rte_flow_query_update() fetches the actual counter
> values from the device using underlying software and/or hardware and
> stores obtained data into the counter objects within PMD.
> 
> The application can get the stored data by invoking the existing
> rte_flow_query() method with specifying the new introduced flag
> 'read_cached'. Such approach is compatible with the current
> implementation, improves the performance and requires the minor changes
> from applications.
> 
> Let's assume we have an array of flows and attached counters.
> If application wants to read them it does something like that:
> 
> foreach(flow) {                                 // compatible mode
>         rte_flow_query(flow, flow_counters[]);  // no read_cached set
> }                                               // doing as previously
> 
> With counter batch query implemented application can do the following:
> 
>                                                 // new query mode
> rte_flow_query_update(group_id of interest);    // actual query here
> foreach(flow) {                                 // as single call
>         rte_flow_query(flow, flow_counters[]);  // read_cached flag set
> }                                               // read stored data
>                                                 // no underlying calls
> 
> For some devices implementation of rte_flow_query_update() may require
> a lot of preparations before performing the actual query. If batch is
> permanent and assumed not to be changed frequently the preparations can
> be cached internally by implementation. By setting the
> RTE_FLOW_QUERY_FLAG_PERMANENT flag application gives the hint to
> PMD that batch is assumed to be long-term and allows to optimize the
> succesive calls of rte_flow_query_update() for the same group_id on given
> device.
> 
> If permanent batch is subject to change (occurs an adding or removing the
> counter with specified batch id) the PMD should free all resources, internally
> allocated for batch query optimization.
> 
> If RTE_FLOW_QUERY_FLAG_PERMANENT is not set,
> rte_flow_query_update() should free the resources allocated (including
> ones done in previous calls, if any) for batch query optimization with given
> group_id.
> 
> Signed-off-by: Viacheslav Ovsiienko <viacheslavo at mellanox.com>
> ---
>  doc/guides/prog_guide/rte_flow.rst | 76
> ++++++++++++++++++++++++++++----------
>  lib/librte_ethdev/rte_flow.h       | 59 ++++++++++++++++++++++++++++-
>  2 files changed, 113 insertions(+), 22 deletions(-)
> 
> diff --git a/doc/guides/prog_guide/rte_flow.rst
> b/doc/guides/prog_guide/rte_flow.rst
> index b305a72..84b3b67 100644
> --- a/doc/guides/prog_guide/rte_flow.rst
> +++ b/doc/guides/prog_guide/rte_flow.rst
> @@ -1500,6 +1500,10 @@ action must specify a unique id.
>  Counters can be retrieved and reset through ``rte_flow_query()``, see
> ``struct rte_flow_query_count``.
> 
> +Counters can be assigned with group_id, all counters with matched
> +group_id on the same port are grouped into batch and can be queried
> +from device using the single call of rte_flow_query_update()
> +
>  The shared flag indicates whether the counter is unique to the flow rule the
> action is specified with, or whether it is a shared counter.
> 
> @@ -1515,13 +1519,15 @@ to all ports within that switch domain.
> 
>  .. table:: COUNT
> 
> -   +------------+---------------------+
> -   | Field      | Value               |
> -   +============+=====================+
> -   | ``shared`` | shared counter flag |
> -   +------------+---------------------+
> -   | ``id``     | counter id          |
> -   +------------+---------------------+
> +   +--------------+----------------------+
> +   | Field        | Value                |
> +   +==============+======================+
> +   | ``shared``   | shared counter flag  |
> +   +--------------+----------------------+
> +   | ``id``       | counter id           |
> +   +--------------+----------------------+
> +   | ``group_id`` | batch id             |
> +   +--------------+----------------------+
> 
>  Query structure to retrieve and reset flow rule counters:
> 
> @@ -1529,19 +1535,21 @@ Query structure to retrieve and reset flow rule
> counters:
> 
>  .. table:: COUNT query
> 
> -   +---------------+-----+-----------------------------------+
> -   | Field         | I/O | Value                             |
> -
> +===============+=====+===================================
> +
> -   | ``reset``     | in  | reset counter after query         |
> -   +---------------+-----+-----------------------------------+
> -   | ``hits_set``  | out | ``hits`` field is set             |
> -   +---------------+-----+-----------------------------------+
> -   | ``bytes_set`` | out | ``bytes`` field is set            |
> -   +---------------+-----+-----------------------------------+
> -   | ``hits``      | out | number of hits for this rule      |
> -   +---------------+-----+-----------------------------------+
> -   | ``bytes``     | out | number of bytes through this rule |
> -   +---------------+-----+-----------------------------------+
> +   +-----------------+-----+-------------------------------------+
> +   | Field           | I/O | Value                               |
> +
> +=================+=====+=================================
> ====+
> +   | ``reset``       | in  | reset counter after query           |
> +   +-----------------+-----+-------------------------------------+
> +   | ``hits_set``    | out | ``hits`` field is set               |
> +   +-----------------+-----+-------------------------------------+
> +   | ``bytes_set``   | out | ``bytes`` field is set              |
> +   +-----------------+-----+-------------------------------------+
> +   | ``read_cached`` | in  | read cached data instead of device  |
> +   +-----------------+-----+-------------------------------------+
> +   | ``hits``        | out | number of hits for this rule        |
> +   +-----------------+-----+-------------------------------------+
> +   | ``bytes``       | out | number of bytes through this rule   |
> +   +-----------------+-----+-------------------------------------+
> 
>  Action: ``RSS``
>  ^^^^^^^^^^^^^^^
> @@ -2288,6 +2296,34 @@ Return values:
> 
>  - 0 on success, a negative errno value otherwise and ``rte_errno`` is set.
> 
> +Batch Query
> +~~~~~~~~~~~
> +
> +Query a batch of existing flow rules.
> +
> +This function allows retrieving flow-specific data such as counters
> +belonging to the different flows on the given port in single batch
> +query call.
> +
> +.. code-block:: c
> +
> +   int
> +   rte_flow_query_update(uint16_t port_id,
> +		         uint32_t group_id,
> +		         uint32_t flags);
> +
> +Arguments:
> +
> +- ``port_id``: port identifier of Ethernet device.
> +- ``group_id``: batch to query, specifies the group of actions.
> +- ``flags``: can be combination of ``RTE_FLOW_QUERY_FLAG_RESET``
> +  and ``RTE_FLOW_QUERY_FLAG_PERMANENT`` values
> +
> +Return values:
> +
> +- 0 on success, a negative errno value otherwise and ``rte_errno`` is set.
> +
> +
>  Isolated mode
>  -------------
> 
> diff --git a/lib/librte_ethdev/rte_flow.h b/lib/librte_ethdev/rte_flow.h index
> f8ba71c..0cbc8fa 100644
> --- a/lib/librte_ethdev/rte_flow.h
> +++ b/lib/librte_ethdev/rte_flow.h
> @@ -1561,10 +1561,14 @@ struct rte_flow_action_queue {
>   * Counters can be retrieved and reset through ``rte_flow_query()``, see
>   * ``struct rte_flow_query_count``.
>   *
> + * Counters can be assigned with group_id, all counters with matched
> + group_id
> + * on the same port are grouped into batch and can be queried from
> + device
> + * using the single call of rte_flow_query_update()
> + *
>   * The shared flag indicates whether the counter is unique to the flow rule
> the
>   * action is specified with, or whether it is a shared counter.
>   *
> - * For a count action with the shared flag set, then then a global device
> + * For a count action with the shared flag set, then a global device
>   * namespace is assumed for the counter id, so that any matched flow rules
> using
>   * a count action with the same counter id on the same port will contribute
> to
>   * that counter.
> @@ -1576,6 +1580,7 @@ struct rte_flow_action_count {
>  	uint32_t shared:1; /**< Share counter ID with other flow rules. */
>  	uint32_t reserved:31; /**< Reserved, must be zero. */
>  	uint32_t id; /**< Counter ID. */
> +	uint32_t group_id; /**< ID of batch that counter belongs to */
>  };
> 
>  /**
> @@ -1587,7 +1592,8 @@ struct rte_flow_query_count {
>  	uint32_t reset:1; /**< Reset counters after query [in]. */
>  	uint32_t hits_set:1; /**< hits field is set [out]. */
>  	uint32_t bytes_set:1; /**< bytes field is set [out]. */
> -	uint32_t reserved:29; /**< Reserved, must be zero [in, out]. */
> +	uint32_t read_cached:1; /**< read stored data instead of device [in].
> */
> +	uint32_t reserved:28; /**< Reserved, must be zero [in, out]. */
>  	uint64_t hits; /**< Number of hits for this rule [out]. */
>  	uint64_t bytes; /**< Number of bytes through this rule [out]. */  };
> @@ -2094,6 +2100,55 @@ struct rte_flow *
>  	       struct rte_flow_error *error);
> 
>  /**
> + * Query a batch of existing flow rules.
> + *
> + * This function allows retrieving flow-specific data such as counters
> + * belonging to the different flows on the given port in single batch
> + * query call.
> + *
> + * \see RTE_FLOW_ACTION_TYPE_COUNT
> + *
> + * @param port_id
> + *   Port identifier of Ethernet device.
> + * @param group_id
> + *   Batch identifier, specifies the group of actions to be queried.
> + * @param flags
> + *   RTE_FLOW_QUERY_FLAG_RESET
> + *   RTE_FLOW_QUERY_FLAG_PERMANENT
> + *
> + * @return
> + *   0 on success, a negative errno value otherwise and rte_errno is set.
> + */
> +int
> +rte_flow_query_update(uint16_t port_id,
> +		      uint32_t group_id,
> +		      uint32_t flags);
> +
> +#define RTE_FLOW_QUERY_FLAG_RESET (1 << 0) /**< Reset counters after
> +query */
> +/**
> + * For some devices implementation of rte_flow_query_update() may
> +require
> + * a lot of preparations before performing the actual query. If batch
> +is
> + * permanent and assumed not to be changed frequently the preparations
> + * can be cached internally by implementation. By setting the
> + * RTE_FLOW_QUERY_FLAG_PERMANENT flag application gives the hint to
> PMD
> + * that batch is assumed to be long-term and allows to optimize the
> + * succesive calls of rte_flow_query_update() for the same group_id
> + * on given device.
> + *
> + * If permanent batch is subject to change (occurs an adding or
> + * removing the action with specified batch id) the PMD should free all
> + * resources, internally allocated for batch query optimization
> + *
> + * If RTE_FLOW_QUERY_FLAG_PERMANENT is not set,
> rte_flow_query_update()
> + * should free the resources (if any) early allocated for batch query
> + * optimization with given group_id.
> + *
> + */
> +#define RTE_FLOW_QUERY_FLAG_PERMANENT (1 << 1) /**< Assume
> batch
> +permanent */
> +
> +/**
>   * Restrict ingress traffic to the defined flow rules.
>   *
>   * Isolated mode guarantees that all ingress traffic comes from defined flow
> --
> 1.8.3.1