[PATCH v7] ethdev: add template table resize API

[Ferruh Yigit]

On 2/14/2024 2:32 PM, Gregory Etelson wrote:
> Template table creation API sets table flows capacity.
> If application needs more flows then the table was designed for,
> the following procedures must be completed:
> 1. Create a new template table with larger flows capacity.
> 2. Re-create existing flows in the new table and delete flows from
>    the original table.
> 3. Destroy original table.
> 
> Application cannot always execute that procedure:
> * Port may not have sufficient resources to allocate a new table
>   while maintaining original table.
> * Application may not have existing flows "recipes" to re-create
>   flows in a new table.
> 
> The patch defines a new API that allows application to resize
> existing template table:
> 
> * Resizable template table must be created with the
>   RTE_FLOW_TABLE_SPECIALIZE_RESIZABLE_TABLE bit set.
> 
> * Application resizes existing table with the
>   `rte_flow_template_table_resize()` function call.
>   The table resize procedure updates the table maximal flow number
>   only. Other table attributes are not affected by the table resize.
>   ** The table resize procedure must not interrupt
>      existing table flows operations in hardware.
>   ** The table resize procedure must not alter flow handles held by
>      application.
> 
> * After `rte_flow_template_table_resize()` returned, application must
>   update table flow rules by calling
>   `rte_flow_async_update_resized()`.
>   The call reconfigures internal flow resources for the new table
>   configuration.
>   The flow update must not interrupt hardware flow operations.
> 
> * After table flows were updated, application must call
>   `rte_flow_template_table_resize_complete()`.
>   The function releases PMD resources related to the original
>   table.
>   Application can start new table resize after
>   `rte_flow_template_table_resize_complete()` returned.
> 
> Testpmd commands:
> 
> * Create resizable template table
> flow template_table <port-id> create table_id <tbl-id> resizable \
>   [transfer|ingress|egres] group <group-id> \
>   rules_number <initial table capacity> \
>   pattern_template  <pt1> [ pattern_template <pt2> [ ... ]] \
>   actions_template  <at1> [ actions_template <at2> [ ... ]]
> 
> * Resize table:
> flow template_table <tbl-id> resize table_resize_id <tbl-id> \
>   table_resize_rules_num <new table capacity>
> 
> * Queue a flow update:
> flow queue <port-id> update_resized <tbl-id> rule <flow-id>
> 
> * Complete table resize:
> flow template_table <port-id> resize_complete table <tbl-id>
> 
> Signed-off-by: Gregory Etelson <getelson at nvidia.com>
> Acked-by: Ori Kam <orika at nvidia.com>
> Acked-by: Ferruh Yigit <ferruh.yigit at amd.com>
> ---
> v2: Update the patch comment.
>     Add table resize commands to testpmd user guide.
> v3: Rename RTE_FLOW_TABLE_SPECIALIZE_RESIZABLE macro.
> v4: Remove inline.
>     Add use case to rte_flow.rst.
> v5: Update API guide in the rte_flow.rst.
> v6: Update docs.
> v7: More API doc updates.
> ---
>  app/test-pmd/cmdline_flow.c                 |  86 +++++++++++++-
>  app/test-pmd/config.c                       | 102 +++++++++++++++++
>  app/test-pmd/testpmd.h                      |   6 +
>  doc/guides/howto/rte_flow.rst               |  88 +++++++++++++++
>  doc/guides/rel_notes/release_24_03.rst      |  12 ++
>  doc/guides/testpmd_app_ug/testpmd_funcs.rst |  15 ++-
>  lib/ethdev/ethdev_trace.h                   |  33 ++++++
>  lib/ethdev/ethdev_trace_points.c            |   9 ++
>  lib/ethdev/rte_flow.c                       |  77 +++++++++++++
>  lib/ethdev/rte_flow.h                       | 119 ++++++++++++++++++++
>  lib/ethdev/rte_flow_driver.h                |  15 +++
>  lib/ethdev/version.map                      |   6 +
>  12 files changed, 562 insertions(+), 6 deletions(-)
> 

Having conflict while applying the patch, can you please rebase it on
latest 'next-net'?

<...>

> diff --git a/doc/guides/rel_notes/release_24_03.rst b/doc/guides/rel_notes/release_24_03.rst
> index 39088da303..ec4d317af7 100644
> --- a/doc/guides/rel_notes/release_24_03.rst
> +++ b/doc/guides/rel_notes/release_24_03.rst
> @@ -55,6 +55,18 @@ New Features
>       Also, make sure to start the actual text at the margin.
>       =======================================================
>  
> +* **Added  API to change template table flows capacity.**
> +
> +  * ``RTE_FLOW_TABLE_SPECIALIZE_RESIZABLE_TABLE`` table configuration bit.
> +    Set when template must be created with the resizable property.
> +  * ``rte_flow_template_table_resizable()``.
> +    Query wheather template table can be resized.
>

s/wheather/whether/

<...>

> +/**
> + * @warning
> + * @b EXPERIMENTAL: this API may change without prior notice.
> + *
> + * Query whether a table can be resized.
> + *
> + * @param port_id
> + *    Port identifier of Ethernet device.
> + * @param tbl_attr
> + *    Template table.
> + *
> + * @return
> + *    True if the table can be resized.
> + */
> +__rte_experimental
> +bool
> +rte_flow_template_table_resizable
> +	(__rte_unused uint16_t port_id,
> +	 const struct rte_flow_template_table_attr *tbl_attr);
> +

Syntax above is odd, why not move parenthesis to first line.

<...>

>  /**
>   * @warning
>   * @b EXPERIMENTAL: this API may change without prior notice.
> @@ -6750,6 +6774,101 @@ rte_flow_calc_table_hash(uint16_t port_id, const struct rte_flow_template_table
>  			 const struct rte_flow_item pattern[], uint8_t pattern_template_index,
>  			 uint32_t *hash, struct rte_flow_error *error);
>  
> +/**
> + * @warning
> + * @b EXPERIMENTAL: this API may change without prior notice.
> + *
> + * Update template table for new flow rules capacity.
> + *
> + * @param port_id
> + *   Port identifier of Ethernet device.
> + * @param table
> + *   Template table to modify.
> + * @param nb_rules
> + *   New flow rules capacity.
> + * @param error
> + *   Perform verbose error reporting if not NULL.
> + *   PMDs initialize this structure in case of error only.
> + *
> + * @return
> + *   - (0) if success.
> + *   - (-ENODEV) if *port_id* invalid.
> + *   - (-ENOTSUP) if underlying device does not support this functionality.
> + *   - (-EINVAL) if *table* is not resizable or
> + *               *table* resize to *nb_rules* is not supported or
> + *               unrecoverable *table* error.
> + */
> +__rte_experimental
> +int
> +rte_flow_template_table_resize(uint16_t port_id,
> +			       struct rte_flow_template_table *table,
> +			       uint32_t nb_rules,
> +			       struct rte_flow_error *error);
> +/**
> + * @warning
> + * @b EXPERIMENTAL: this API may change without prior notice.
> + *
> + * Update flow for the new template table configuration after table resize.
> + * Must be called for each *rule* created before and after *table* resize.
>

This is different than previous version, I just want to confirm if this
is intentional.

Rules created *before* resize must be updated.
Rules created *after* resize not need to be updated, but if API accepts
them and just returns a quick success, this helps user and even user may
prefer to not keep track of flows as before and after resize, in a good
time user can call update for all flows.

But I am not clear why API is saying all flows (before or after) *must*
be updated?
If user is already keeping record of flows *before* resize, why not let
app to update only those flows?

Is the enforcing update of all flows done intentionally, or is it just
wording error?

<...>

> +/**
> + * @warning
> + * @b EXPERIMENTAL: this API may change without prior notice.
> + *
> + * Resume normal operational mode after table was resized and
> + * table rules were updated for the new table configuration.
> + *
> + * @param port_id
> + *   Port identifier of Ethernet device.
> + * @param table
> + *   Template table that undergoing resize operation.
> + * @param error
> + *   Perform verbose error reporting if not NULL.
> + *   PMDs initialize this structure in case of error only.
> + *
> + * @return
> + *   - (0) if success.
> + *   - (-ENODEV) if *port_id* invalid.
> + *   - (-ENOTSUP) if underlying device does not support this functionality.
> + *   - (-EINVAL) if there are rules that were not updated or
> + *               *table* cannot complete table resize,
> + *               unrecoverable error.
> + */
> +__rte_experimental
> +int
> +rte_flow_template_table_resize_complete(uint16_t port_id,
> +					struct rte_flow_template_table *table,
> +					struct rte_flow_error *error);

I think I asked this before but perhaps missed the response,
it is possible that user missed to update all flows and called this API,
it will return -EINVAL, in this case it is not clear for user if there
is a unrecoverable error or just need to update more flows.

What do you think to send a different error for the case that there are
flows not updated, so user can action on this information.