[PATCH v3 01/10] ethdev: introduce flow pre-configuration hints

[Jerin Jacob]

On Sun, Feb 6, 2022 at 8:56 AM Alexander Kozyrev <akozyrev at nvidia.com> wrote:
>
> The flow rules creation/destruction at a large scale incurs a performance
> penalty and may negatively impact the packet processing when used
> as part of the datapath logic. This is mainly because software/hardware
> resources are allocated and prepared during the flow rule creation.
>
> In order to optimize the insertion rate, PMD may use some hints provided
> by the application at the initialization phase. The rte_flow_configure()
> function allows to pre-allocate all the needed resources beforehand.
> These resources can be used at a later stage without costly allocations.
> Every PMD may use only the subset of hints and ignore unused ones or
> fail in case the requested configuration is not supported.
>
> The rte_flow_info_get() is available to retrieve the information about
> supported pre-configurable resources. Both these functions must be called
> before any other usage of the flow API engine.
>
> Signed-off-by: Alexander Kozyrev <akozyrev at nvidia.com>
> ---
>  doc/guides/prog_guide/rte_flow.rst     | 37 ++++++++++++
>  doc/guides/rel_notes/release_22_03.rst |  4 ++
>  lib/ethdev/rte_flow.c                  | 40 +++++++++++++
>  lib/ethdev/rte_flow.h                  | 82 ++++++++++++++++++++++++++
>  lib/ethdev/rte_flow_driver.h           | 10 ++++
>  lib/ethdev/version.map                 |  4 ++
>  6 files changed, 177 insertions(+)
>
> diff --git a/doc/guides/prog_guide/rte_flow.rst b/doc/guides/prog_guide/rte_flow.rst
> index b4aa9c47c2..5b4c5dd609 100644
> --- a/doc/guides/prog_guide/rte_flow.rst
> +++ b/doc/guides/prog_guide/rte_flow.rst
> @@ -3589,6 +3589,43 @@ Return values:
>
>  - 0 on success, a negative errno value otherwise and ``rte_errno`` is set.
>
> +Flow engine configuration
> +-------------------------
> +
> +Configure flow API management.
> +
> +An application may provide some hints at the initialization phase about
> +rules engine configuration and/or expected flow rules characteristics.

IMO, We can explicitly remove _hint_ in the documentation.
When we add new paramers to configure, it may not be hit.

> +These hints may be used by PMD to pre-allocate resources and configure NIC.

hints->parameters

> +
> +Configuration
> +~~~~~~~~~~~~~
> +
> +This function performs the flow API management configuration and
> +pre-allocates needed resources beforehand to avoid costly allocations later.
> +Hints about the expected number of counters or meters in an application,
> +for example, allow PMD to prepare and optimize NIC memory layout in advance.
> +``rte_flow_configure()`` must be called before any flow rule is created,
> +but after an Ethernet device is configured.
> +
> +.. code-block:: c
> +
> +   int
> +   rte_flow_configure(uint16_t port_id,
> +                     const struct rte_flow_port_attr *port_attr,
> +                     struct rte_flow_error *error);
> +
> +Information about resources that can benefit from pre-allocation can be
> +retrieved via ``rte_flow_info_get()`` API. It returns the maximum number
> +of pre-configurable resources for a given port on a system.
> +
> +.. code-block:: c
> +
> +   int
> +   rte_flow_info_get(uint16_t port_id,
> +                     struct rte_flow_port_attr *port_attr,
> +                     struct rte_flow_error *error);
> +
>  .. _flow_isolated_mode:
>
>  Flow isolated mode
> diff --git a/doc/guides/rel_notes/release_22_03.rst b/doc/guides/rel_notes/release_22_03.rst
> index bf2e3f78a9..8593db3f6a 100644
> --- a/doc/guides/rel_notes/release_22_03.rst
> +++ b/doc/guides/rel_notes/release_22_03.rst
> @@ -55,6 +55,10 @@ New Features
>       Also, make sure to start the actual text at the margin.
>       =======================================================
>
> +* ethdev: Added ``rte_flow_configure`` API to configure Flow Management
> +  engine, allowing to pre-allocate some resources for better performance.
> +  Added ``rte_flow_info_get`` API to retrieve pre-configurable resources.
> +
>  * **Updated AF_XDP PMD**
>
>    * Added support for libxdp >=v1.2.2.
> diff --git a/lib/ethdev/rte_flow.c b/lib/ethdev/rte_flow.c
> index a93f68abbc..e7e6478bed 100644
> --- a/lib/ethdev/rte_flow.c
> +++ b/lib/ethdev/rte_flow.c
> @@ -1391,3 +1391,43 @@ rte_flow_flex_item_release(uint16_t port_id,
>         ret = ops->flex_item_release(dev, handle, error);
>         return flow_err(port_id, ret, error);
>  }
> +
> +int
> +rte_flow_info_get(uint16_t port_id,
> +                 struct rte_flow_port_attr *port_attr,
> +                 struct rte_flow_error *error)
> +{
> +       struct rte_eth_dev *dev = &rte_eth_devices[port_id];
> +       const struct rte_flow_ops *ops = rte_flow_ops_get(port_id, error);
> +
> +       if (unlikely(!ops))
> +               return -rte_errno;
> +       if (likely(!!ops->info_get)) {
> +               return flow_err(port_id,
> +                               ops->info_get(dev, port_attr, error),
> +                               error);
> +       }
> +       return rte_flow_error_set(error, ENOTSUP,
> +                                 RTE_FLOW_ERROR_TYPE_UNSPECIFIED,
> +                                 NULL, rte_strerror(ENOTSUP));
> +}
> +
> +int
> +rte_flow_configure(uint16_t port_id,
> +                  const struct rte_flow_port_attr *port_attr,
> +                  struct rte_flow_error *error)
> +{
> +       struct rte_eth_dev *dev = &rte_eth_devices[port_id];
> +       const struct rte_flow_ops *ops = rte_flow_ops_get(port_id, error);
> +
> +       if (unlikely(!ops))
> +               return -rte_errno;
> +       if (likely(!!ops->configure)) {
> +               return flow_err(port_id,
> +                               ops->configure(dev, port_attr, error),
> +                               error);
> +       }
> +       return rte_flow_error_set(error, ENOTSUP,
> +                                 RTE_FLOW_ERROR_TYPE_UNSPECIFIED,
> +                                 NULL, rte_strerror(ENOTSUP));
> +}
> diff --git a/lib/ethdev/rte_flow.h b/lib/ethdev/rte_flow.h
> index 1031fb246b..f3c7159484 100644
> --- a/lib/ethdev/rte_flow.h
> +++ b/lib/ethdev/rte_flow.h
> @@ -4853,6 +4853,88 @@ rte_flow_flex_item_release(uint16_t port_id,
>                            const struct rte_flow_item_flex_handle *handle,
>                            struct rte_flow_error *error);
>
> +/**
> + * @warning
> + * @b EXPERIMENTAL: this API may change without prior notice.
> + *
> + * Resource pre-allocation settings.
> + * The zero value means on demand resource allocations only.
> + *
> + */
> +struct rte_flow_port_attr {
> +       /**
> +        * Version of the struct layout, should be 0.
> +        */
> +       uint32_t version;

IMO, it is not concluded to use the version in the public API.


> +       /**
> +        * Number of counter actions pre-configured.
> +        * @see RTE_FLOW_ACTION_TYPE_COUNT
> +        */
> +       uint32_t nb_counters;
> +       /**
> +        * Number of aging flows actions pre-configured.
> +        * @see RTE_FLOW_ACTION_TYPE_AGE
> +        */
> +       uint32_t nb_aging_flows;
> +       /**
> +        * Number of traffic metering actions pre-configured.
> +        * @see RTE_FLOW_ACTION_TYPE_METER
> +        */
> +       uint32_t nb_meters;
> +};
> +
> +/**
> + * @warning
> + * @b EXPERIMENTAL: this API may change without prior notice.
> + *
> + * Retrieve configuration attributes supported by the port.
> + *
> + * @param port_id
> + *   Port identifier of Ethernet device.
> + * @param[in] port_attr

By default all parameters are "in", no need for additional "in" for just input.

> + *   Port configuration attributes.
> + * @param[out] error
> + *   Perform verbose error reporting if not NULL.
> + *   PMDs initialize this structure in case of error only.
> + *
> + * @return
> + *   0 on success, a negative errno value otherwise and rte_errno is set.
> + */
> +__rte_experimental
> +int
> +rte_flow_info_get(uint16_t port_id,
> +                 struct rte_flow_port_attr *port_attr,
> +                 struct rte_flow_error *error);
> +
> +/**
> + * @warning
> + * @b EXPERIMENTAL: this API may change without prior notice.
> + *
> + * Pre-configure the port's flow API engine.

IMO, s/ Pre-configure/Configure/

> + *
> + * This API can only be invoked before the application
> + * starts using the rest of the flow library functions.
> + *
> + * The API can be invoked multiple times to change the
> + * settings. The port, however, may reject the changes.
> + *
> + * @param port_id
> + *   Port identifier of Ethernet device.
> + * @param[in] port_attr
> + *   Port configuration attributes.

IMO, we need to have comments, the values should be <= the values
got it from rte_flow_info_get().

Also 0 representations the default value.

> + * @param[out] error
> + *   Perform verbose error reporting if not NULL.
> + *   PMDs initialize this structure in case of error only.
> + *
> + * @return
> + *   0 on success, a negative errno value otherwise and rte_errno is set.



> + */
> +__rte_experimental
> +int
> +rte_flow_configure(uint16_t port_id,
> +                  const struct rte_flow_port_attr *port_attr,
> +                  struct rte_flow_error *error);
> +
>  #ifdef __cplusplus
>  }
>  #endif
> diff --git a/lib/ethdev/rte_flow_driver.h b/lib/ethdev/rte_flow_driver.h
> index f691b04af4..503700aec4 100644
> --- a/lib/ethdev/rte_flow_driver.h
> +++ b/lib/ethdev/rte_flow_driver.h
> @@ -152,6 +152,16 @@ struct rte_flow_ops {
>                 (struct rte_eth_dev *dev,
>                  const struct rte_flow_item_flex_handle *handle,
>                  struct rte_flow_error *error);
> +       /** See rte_flow_info_get() */
> +       int (*info_get)
> +               (struct rte_eth_dev *dev,
> +                struct rte_flow_port_attr *port_attr,
> +                struct rte_flow_error *err);
> +       /** See rte_flow_configure() */
> +       int (*configure)
> +               (struct rte_eth_dev *dev,
> +                const struct rte_flow_port_attr *port_attr,
> +                struct rte_flow_error *err);
>  };
>
>  /**
> diff --git a/lib/ethdev/version.map b/lib/ethdev/version.map
> index 1f7359c846..59785c3634 100644
> --- a/lib/ethdev/version.map
> +++ b/lib/ethdev/version.map
> @@ -256,6 +256,10 @@ EXPERIMENTAL {
>         rte_flow_flex_item_create;
>         rte_flow_flex_item_release;
>         rte_flow_pick_transfer_proxy;
> +
> +       # added in 22.03
> +       rte_flow_info_get;
> +       rte_flow_configure;
>  };
>
>  INTERNAL {
> --
> 2.18.2
>