[dpdk-dev] [PATCH v4 2/2] ethdev: add traffic management API

[Manoharan, Balasubramanian]

I am fine with this proposal.

Acked-by: Balasubramanian Manoharan
<balasubramanian.manoharan at caviumnetworks.com>

> On 31-May-2017, at 7:15 PM, Jacob, Jerin <Jerin.JacobKollanukkaran at cavium.com> wrote:
> 
> -----Original Message-----
>> Date: Fri, 19 May 2017 18:12:52 +0100
>> From: Cristian Dumitrescu <cristian.dumitrescu at intel.com>
>> To: dev at dpdk.org
>> CC: thomas.monjalon at 6wind.com, jerin.jacob at caviumnetworks.com,
>> balasubramanian.manoharan at cavium.com, hemant.agrawal at nxp.com,
>> shreyansh.jain at nxp.com
>> Subject: [PATCH v4 2/2] ethdev: add traffic management API
>> X-Mailer: git-send-email 2.7.4
>> 
>> This patch introduces the generic ethdev API for the traffic manager
>> capability, which includes: hierarchical scheduling, traffic shaping,
>> congestion management, packet marking.
>> 
>> Main features:
>> - Exposed as ethdev plugin capability (similar to rte_flow)
>> - Capability query API per port, per level and per node
>> - Scheduling algorithms: Strict Priority (SP), Weighed Fair Queuing (WFQ)
>> - Traffic shaping: single/dual rate, private (per node) and shared (by
>>  multiple nodes) shapers
>> - Congestion management for hierarchy leaf nodes: algorithms of tail drop,
>>  head drop, WRED; private (per node) and shared (by multiple nodes) WRED
>>  contexts
>> - Packet marking: IEEE 802.1q (VLAN DEI), IETF RFC 3168 (IPv4/IPv6 ECN for
>>  TCP and SCTP), IETF RFC 2597 (IPv4 / IPv6 DSCP)
>> 
>> Signed-off-by: Cristian Dumitrescu <cristian.dumitrescu at intel.com>
> 
> IMO, With this version, It is reached to a reasonable shape where we can start
> using it as a base for next-tm if there are no other reviewers for this
> feature.
> 
> Two major comments,
> 1) IMO, We don't need a separate API for rte_tm_node_add_check_level()
> and rte_tm_node_add().We can just keep, rte_tm_node_add() and move the
> level check in common code.
> 
> 2) There are a lot of doxygen rendering issues in this document. I will try
> to enumerate them inline. Please cross check the header file with
> "make doc-api-html" output.
> 
> With above changes,
> Acked-by: Jerin Jacob <jerin.jacob at caviumnetworks.com>
> 
> 
>> 
>> MAINTAINERS                            |    4 +
>> lib/librte_ether/Makefile              |    5 +-
>> lib/librte_ether/rte_ether_version.map |   30 +
>> lib/librte_ether/rte_tm.c              |  448 ++++++++
>> lib/librte_ether/rte_tm.h              | 1923 ++++++++++++++++++++++++++++++++
>> lib/librte_ether/rte_tm_driver.h       |  373 +++++++
> 
> Missing doxygen hooks in doc/api/doxy-api-index.md
> 
> [rte_tm]             (@ref rte_tm.h),
> [rte_tm_driver]      (@ref rte_tm_driver.h),
> 
>> 6 files changed, 2782 insertions(+), 1 deletion(-)
>> create mode 100644 lib/librte_ether/rte_tm.c
>> create mode 100644 lib/librte_ether/rte_tm.h
>> create mode 100644 lib/librte_ether/rte_tm_driver.h
>> 
>> diff --git a/MAINTAINERS b/MAINTAINERS
>> index afb4cab..cdaf2ac 100644
>> --- a/MAINTAINERS
>> +++ b/MAINTAINERS
>> @@ -240,6 +240,10 @@ Flow API
>> M: Adrien Mazarguil <adrien.mazarguil at 6wind.com>
>> F: lib/librte_ether/rte_flow*
>> 
>> +Traffic Management API
>> +M: Cristian Dumitrescu <cristian.dumitrescu at intel.com>
>> +F: lib/librte_ether/rte_tm*
> 
> Add next-tm tree here.
> 
>> +
>> Crypto API
>> M: Declan Doherty <declan.doherty at intel.com>
>> F: lib/librte_cryptodev/
>> diff --git a/lib/librte_ether/Makefile b/lib/librte_ether/Makefile
>> index 93fdde1..db692ae 100644
>> --- a/lib/librte_ether/Makefile
>> +++ b/lib/librte_ether/Makefile
>> @@ -1,6 +1,6 @@
>> #   BSD LICENSE
>> #
>> -#   Copyright(c) 2010-2016 Intel Corporation. All rights reserved.
>> +#   Copyright(c) 2010-2017 Intel Corporation. All rights reserved.
> 
> Good to add them the name of other companies who contributed in the
> specification.
> 
>> #   All rights reserved.
>> #
>> #   Redistribution and use in source and binary forms, with or without
>> @@ -45,6 +45,7 @@ LIBABIVER := 6
>> 
>> SRCS-y += rte_ethdev.c
>> SRCS-y += rte_flow.c
>> +SRCS-y += rte_tm.c
>> 
>> +            NULL,                \
>> +            rte_strerror(ENOSYS));        \
>> +                            \
>> +    ops->func;                    \
>> +})
>> +
>> +/* Get number of leaf nodes */
>> +int
>> +rte_tm_get_number_of_leaf_nodes(uint8_t port_id,
>> +    uint32_t *n_leaf_nodes,
>> +    struct rte_tm_error *error)
>> +{
>> +    struct rte_eth_dev *dev = &rte_eth_devices[port_id];
>> +    const struct rte_tm_ops *ops =
>> +        rte_tm_ops_get(port_id, error);
>> +
>> +    if (ops == NULL)
>> +        return -rte_errno;
>> +
>> +    if (n_leaf_nodes == NULL) {
>> +        rte_tm_error_set(error,
>> +            EINVAL,
>> +            RTE_TM_ERROR_TYPE_UNSPECIFIED,
>> +            NULL,
>> +            rte_strerror(EINVAL));
>> +        return -rte_errno;
>> +    }
>> +
>> +    *n_leaf_nodes = dev->data->nb_tx_queues;
>> +    return 0;
>> +}
>> +
>> +/* Check node type (leaf or non-leaf) */
>> +int
>> +rte_tm_node_type_get(uint8_t port_id,
>> +    uint32_t node_id,
>> +    int *is_leaf,
>> +    struct rte_tm_error *error)
>> +{
>> +    struct rte_eth_dev *dev = &rte_eth_devices[port_id];
> 
> leaf node can be detected in the common code itself as it is 0 to
> dev->data->nb_tx_queues.
> 
> 
>> +    return RTE_TM_FUNC(port_id, node_type_get)(dev,
>> +        node_id, is_leaf, error);
>> +}
>> +
>> + *   THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, OR TORT
>> + *   (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE
>> + *   OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE.
>> + */
>> +
>> +#ifndef __INCLUDE_RTE_TM_H__
>> +#define __INCLUDE_RTE_TM_H__
>> +
>> +/**
>> + * @file
>> + * RTE Generic Traffic Manager API
>> + *
>> + * This interface provides the ability to configure the traffic manager in a
>> + * generic way. It includes features such as: hierarchical scheduling,
>> + * traffic shaping, congestion management, packet marking, etc.
>> + */
>> +
>> +#include <stdint.h>
>> +
>> +#ifdef __cplusplus
>> +extern "C" {
>> +#endif
>> +
>> +/**
>> + * Ethernet framing overhead.
>> + *
>> + * Overhead fields per Ethernet frame:
>> + * 1. Preamble:                                            7 bytes;
>> + * 2. Start of Frame Delimiter (SFD):                      1 byte;
>> + * 3. Inter-Frame Gap (IFG):                              12 bytes.
>> + *
>> + * One of the typical values for the *pkt_length_adjust* field of the shaper
>> + * profile.
>> + *
>> + * @see struct rte_tm_shaper_params
>> + *
>> + */
>> +#define RTE_TM_ETH_FRAMING_OVERHEAD                  20
>> +
>> +/**
>> + * Ethernet framing overhead including the Frame Check Sequence (FCS) field.
>> + * Useful when FCS is generated and added at the end of the Ethernet frame on
>> + * TX side without any SW intervention.
>> + *
>> + * One of the typical values for the pkt_length_adjust field of the shaper
>> + * profile.
>> + *
>> + * @see struct rte_tm_shaper_params
>> + */
>> +#define RTE_TM_ETH_FRAMING_OVERHEAD_FCS              24
>> +
>> +/**< Invalid WRED profile ID */
>> +#define RTE_TM_WRED_PROFILE_ID_NONE                  UINT32_MAX
>> +
>> +/**< Invalid shaper profile ID */
>> +#define RTE_TM_SHAPER_PROFILE_ID_NONE                UINT32_MAX
>> +
>> +/**< Node ID for the parent of the root node */
>> +#define RTE_TM_NODE_ID_NULL                          UINT32_MAX
>> +
>> +/**
>> + * Color
>> + */
>> +enum rte_tm_color {
>> +    RTE_TM_GREEN = 0, /**< Green */
>> +    RTE_TM_YELLOW, /**< Yellow */
>> +    RTE_TM_RED, /**< Red */
>> +    RTE_TM_COLORS /**< Number of colors */
>> +};
>> +
>> +/**
>> + * Node statistics counter type
>> + */
>> +enum rte_tm_stats_type {
>> +    /**< Number of packets scheduled from current node. */
>> +    RTE_TM_STATS_N_PKTS = 1 << 0,
>> +
>> +    /**< Number of bytes scheduled from current node. */
>> +    RTE_TM_STATS_N_BYTES = 1 << 1,
>> +
>> +    /**< Number of green packets dropped by current leaf node.  */
>> +    RTE_TM_STATS_N_PKTS_GREEN_DROPPED = 1 << 2,
>> +
>> +    /**< Number of yellow packets dropped by current leaf node.  */
>> +    RTE_TM_STATS_N_PKTS_YELLOW_DROPPED = 1 << 3,
>> +
>> +    /**< Number of red packets dropped by current leaf node.  */
>> +    RTE_TM_STATS_N_PKTS_RED_DROPPED = 1 << 4,
>> +
>> +    /**< Number of green bytes dropped by current leaf node.  */
>> +    RTE_TM_STATS_N_BYTES_GREEN_DROPPED = 1 << 5,
>> +
>> +    /**< Number of yellow bytes dropped by current leaf node.  */
>> +    RTE_TM_STATS_N_BYTES_YELLOW_DROPPED = 1 << 6,
>> +
>> +    /**< Number of red bytes dropped by current leaf node.  */
>> +    RTE_TM_STATS_N_BYTES_RED_DROPPED = 1 << 7,
>> +
>> +    /**< Number of packets currently waiting in the packet queue of current
>> +     * leaf node.
>> +     */
>> +    RTE_TM_STATS_N_PKTS_QUEUED = 1 << 8,
>> +
>> +    /**< Number of bytes currently waiting in the packet queue of current
>> +     * leaf node.
>> +     */
>> +    RTE_TM_STATS_N_BYTES_QUEUED = 1 << 9,
>> +};
>> +
> 
>> + * Node statistics counters
>> + */
>> +struct rte_tm_node_stats {
>> +    /**< Number of packets scheduled from current node. */
>> +    uint64_t n_pkts;
> 
> Incorrect doxygen API HTML rendering. It is rendering as
> "< Number of packets scheduled from current node.
> ^^^
> 
> Looks like comment has to come below the "uint64_t n_pkts". Applicable
> across the header file.
> 
> 
>> +
>> +    /**< Number of bytes scheduled from current node. */
>> +    uint64_t n_bytes;
>> +
>> +    /**< Statistics counters for leaf nodes only. */
>> +    struct {
>> +        /**< Number of packets dropped by current leaf node per each
>> +         * color.
>> +         */
>> +        uint64_t n_pkts_dropped[RTE_TM_COLORS];
>> +
>> +        /**< Number of bytes dropped by current leaf node per each
>> +         * color.
>> +         */
>> +        uint64_t n_bytes_dropped[RTE_TM_COLORS];
>> +
>> +        /**< Number of packets currently waiting in the packet queue of
>> +         * current leaf node.
>> +         */
>> +        uint64_t n_pkts_queued;
>> +
>> +        /**< Number of bytes currently waiting in the packet queue of
>> +         * current leaf node.
>> +         */
>> +        uint64_t n_bytes_queued;
>> +    } leaf;
>> +};
>> +
>> +/**
>> + * Traffic manager dynamic updates
>> + */
>> +enum rte_tm_dynamic_update_type {
>> +    /**< Dynamic parent node update. The new parent node is located on same
>> +     * hierarchy level as the former parent node. Consequently, the node
>> +     * whose parent is changed preserves its hierarchy level.
>> +     */
>> +    RTE_TM_UPDATE_NODE_PARENT_KEEP_LEVEL = 1 << 0,
>> +
>> +    /**< Dynamic parent node update. The new parent node is located on
>> +     * different hierarchy level than the former parent node. Consequently,
>> +     * the node whose parent is changed also changes its hierarchy level.
>> +     */
>> +    RTE_TM_UPDATE_NODE_PARENT_CHANGE_LEVEL = 1 << 1,
>> +
>> +    /**< Dynamic node add/delete. */
>> +    RTE_TM_UPDATE_NODE_ADD_DELETE = 1 << 2,
>> +
>> +    /**< Suspend/resume nodes. */
>> +    RTE_TM_UPDATE_NODE_SUSPEND_RESUME = 1 << 3,
>> +
>> +    /**< Dynamic switch between byte-based and packet-based WFQ weights. */
>> +    RTE_TM_UPDATE_NODE_WFQ_WEIGHT_MODE = 1 << 4,
>> +
>> +    /**< Dynamic update on number of SP priorities. */
>> +    RTE_TM_UPDATE_NODE_N_SP_PRIORITIES = 1 << 5,
>> +
>> +    /**< Dynamic update of congestion management mode for leaf nodes. */
>> +    RTE_TM_UPDATE_NODE_CMAN = 1 << 6,
>> +
>> +    /**< Dynamic update of the set of enabled stats counter types. */
>> +    RTE_TM_UPDATE_NODE_STATS = 1 << 7,
>> +};
>> +
>> +/**
>> + * Traffic manager get number of leaf nodes
>> + *
>> + * Each leaf node sits on on top of a TX queue of the current Ethernet port.
>> + * Therefore, the set of leaf nodes is predefined, their number is always equal
>> + * to N (where N is the number of TX queues configured for the current port)
>> + * and their IDs are 0 .. (N-1).
>> + *
>> + * @param[in] port_id
> 
> [in] can be treated as default to avoid mentioning [in] everywhere.
> 
>> + *   The port identifier of the Ethernet device.
>> + * @param[out] n_leaf_nodes
>> + *   Number of leaf nodes for the current port.
>> + * @param[out] error
>> + *   Error details. Filled in only on error, when not NULL.
>> + * @return
>> + *   0 on success, non-zero error code otherwise.
> 
>> +/**
>> + * Traffic manager node delete
>> + *
>> + * Delete an existing node. This operation fails when this node currently has
>> + * at least one user (i.e. child node).
>> + *
>> + * When called before rte_tm_hierarchy_commit() invocation, this function is
>> + * typically used to define the initial start-up hierarchy for the port.
>> + * Provided that dynamic hierarchy updates are supported by the current port (as
>> + * advertised in the port capability set), this function can be also called
>> + * after the rte_tm_hierarchy_commit() invocation.
>> + *
>> + * @param[in] port_id
>> + *   The port identifier of the Ethernet device.
>> + * @param[in] node_id
>> + *   Node ID. Needs to be valid.
>> + * @param[out] error
>> + *   Error details. Filled in only on error, when not NULL.
>> + * @return
>> + *   0 on success, non-zero error code otherwise.
>> + *
>> + * @see RTE_TM_UPDATE_NODE_ADD_DELETE
>> + */
>> +int
>> +rte_tm_node_delete(uint8_t port_id,
>> +    uint32_t node_id,
>> +    struct rte_tm_error *error);
>> +
>> +/**
>> + * Traffic manager node suspend
>> + *
>> + * Suspend an existing node. While the node is in suspended state, no packet is
>> + * scheduled from this node and its descendants. The node exits the suspended
>> + * state through the node resume operation.
>> + *
>> + * @param[in] port_id
>> + *   The port identifier of the Ethernet device.
>> + * @param[in] node_id
>> + *   Node ID. Needs to be valid.
>> + * @param[out] error
>> + *   Error details. Filled in only on error, when not NULL.
>> + * @return
>> + *   0 on success, non-zero error code otherwise.
>> + *
>> + * @see rte_tm_node_resume()
>> + * @see RTE_TM_UPDATE_NODE_SUSPEND_RESUME
>> + */
>> +int
>> +rte_tm_node_suspend(uint8_t port_id,
>> +    uint32_t node_id,
>> +    struct rte_tm_error *error);
>> +
>> +/**
>> + * Traffic manager node resume
>> + *
>> + * Resume an existing node that is currently in suspended state. The node
>> + * entered the suspended state as result of a previous node suspend operation.
>> + *
>> + * @param[in] port_id
>> + *   The port identifier of the Ethernet device.
>> + * @param[in] node_id
>> + *   Node ID. Needs to be valid.
>> + * @param[out] error
>> + *   Error details. Filled in only on error, when not NULL.
>> + * @return
>> + *   0 on success, non-zero error code otherwise.
>> + *
>> + * @see rte_tm_node_suspend()
>> + * @see RTE_TM_UPDATE_NODE_SUSPEND_RESUME
>> + */
>> +int
>> +rte_tm_node_resume(uint8_t port_id,
>> +    uint32_t node_id,
>> +    struct rte_tm_error *error);
>> +
>> +/**
>> + * Traffic manager hierarchy commit
>> + *
>> + * This function is called during the port initialization phase (before the
>> + * Ethernet port is started) to freeze the start-up hierarchy.
>> + *
>> + * This function typically performs the following steps:
>> + *    a) It validates the start-up hierarchy that was previously defined for the
>> + *       current port through successive rte_tm_node_add() invocations;
>> + *    b) Assuming successful validation, it performs all the necessary port
>> + *       specific configuration operations to install the specified hierarchy on
>> + *       the current port, with immediate effect once the port is started.
>> + *
>> + * This function fails when the currently configured hierarchy is not supported
>> + * by the Ethernet port, in which case the user can abort or try out another
>> + * hierarchy configuration (e.g. a hierarchy with less leaf nodes), which can be
>> + * build from scratch (when *clear_on_fail* is enabled) or by modifying the
>> + * existing hierarchy configuration (when *clear_on_fail* is disabled).
>> + *
>> + * Note that this function can still fail due to other causes (e.g. not enough
>> + * memory available in the system, etc), even though the specified hierarchy is
>> + * supported in principle by the current port.
>> + *
>> + * @param[in] port_id
>> + *   The port identifier of the Ethernet device.
>> + * @param[in] clear_on_fail
>> + *   On function call failure, hierarchy is cleared when this parameter is
>> + *   non-zero and preserved when this parameter is equal to zero.
>> + * @param[out] error
>> + *   Error details. Filled in only on error, when not NULL.
>> + * @return
>> + *   0 on success, non-zero error code otherwise.
>> + *
>> + * @see rte_tm_node_add()
>> + * @see rte_tm_node_delete()
>> + */
>> +int
>> +rte_tm_hierarchy_commit(uint8_t port_id,
>> +    int clear_on_fail,
>> +    struct rte_tm_error *error);
>> +
>> +/**
>> + * Traffic manager node parent update
>> + *
>> + * Restriction for root node: its parent cannot be changed.
>> + *
>> + * This function can only be called after the rte_tm_hierarchy_commit()
>> + * invocation. Its success depends on the port support for this operation, as
>> + * advertised through the port capability set.
>> + *
>> + * @param[in] port_id
>> + *   The port identifier of the Ethernet device.
>> + * @param[in] node_id
>> + *   Node ID. Needs to be valid.
>> + * @param[in] parent_node_id
>> + *   Node ID for the new parent. Needs to be valid.
>> + * @param[in] priority
>> + *   Node priority. The highest node priority is zero. Used by the SP algorithm
>> + *   running on the parent of the current node for scheduling this child node.
>> + * @param[in] weight
>> + *   Node weight. The node weight is relative to the weight sum of all siblings
>> + *   that have the same priority. The lowest weight is zero. Used by the WFQ
>> + *   algorithm running on the parent of the current node for scheduling this
>> + *   child node.
>> + * @param[out] error
>> + *   Error details. Filled in only on error, when not NULL.
>> + * @return
>> + *   0 on success, non-zero error code otherwise.
>> + *
>> + * @see RTE_TM_UPDATE_NODE_PARENT_KEEP_LEVEL
>> + * @see RTE_TM_UPDATE_NODE_PARENT_CHANGE_LEVEL
>> + */
>> +int
>> +rte_tm_node_parent_update(uint8_t port_id,
>> +    uint32_t node_id,
>> +    uint32_t parent_node_id,
>> +    uint32_t priority,
>> +    uint32_t weight,
>> +    struct rte_tm_error *error);
>> +
>> +/**
>> + * Traffic manager node private shaper update
>> + *
>> + * Restriction for the root node: its private shaper profile needs to be valid
>> + * and single rate.
>> + *
>> + * @param[in] port_id
>> + *   The port identifier of the Ethernet device.
>> + * @param[in] node_id
>> + *   Node ID. Needs to be valid.
>> + * @param[in] shaper_profile_id
>> + *   Shaper profile ID for the private shaper of the current node. Needs to be
>> + *   either valid shaper profile ID or RTE_TM_SHAPER_PROFILE_ID_NONE, with
>> + *   the latter disabling the private shaper of the current node.
>> + * @param[out] error
>> + *   Error details. Filled in only on error, when not NULL.
>> + * @return
>> + *   0 on success, non-zero error code otherwise.
> 
> Missing the @see to point the capability.
> 
> 
>> + */
>> +int
>> +rte_tm_node_shaper_update(uint8_t port_id,
>> +    uint32_t node_id,
>> +    uint32_t shaper_profile_id,
>> +    struct rte_tm_error *error);
>> +
>> +/**
>> + * Traffic manager node shared shapers update
>> + *
>> + * Restriction for root node: cannot use any shared rate shapers.
>> + *
>> + * @param[in] port_id
>> + *   The port identifier of the Ethernet device.
>> + * @param[in] node_id
>> + *   Node ID. Needs to be valid.
>> + * @param[in] shared_shaper_id
>> + *   Shared shaper ID. Needs to be valid.
>> + * @param[in] add
>> + *   Set to non-zero value to add this shared shaper to current node or to zero
>> + *   to delete this shared shaper from current node.
>> + * @param[out] error
>> + *   Error details. Filled in only on error, when not NULL.
>> + * @return
>> + *   0 on success, non-zero error code otherwise.
> 
> Missing the @see to point the capability.
> 
>> + */
>> +int
>> +rte_tm_node_shared_shaper_update(uint8_t port_id,
>> +    uint32_t node_id,
>> +    uint32_t shared_shaper_id,
>> +    int add,
>> +    struct rte_tm_error *error);
>> +
>> +/**
>> + * Traffic manager node enabled statistics counters update
>> + *
>> + * @param[in] port_id
>> + *   The port identifier of the Ethernet device.
>> + * @param[in] node_id
>> + *   Node ID. Needs to be valid.
>> + * @param[in] stats_mask
>> + *   Mask of statistics counter types to be enabled for the current node. This
>> + *   needs to be a subset of the statistics counter types available for the
>> + *   current node. Any statistics counter type not included in this set is to
>> + *   be disabled for the current node.
>> + * @param[out] error
>> + *   Error details. Filled in only on error, when not NULL.
>> + * @return
>> + *   0 on success, non-zero error code otherwise.
>> + *
>> + * @see enum rte_tm_stats_type
>> + * @see RTE_TM_UPDATE_NODE_STATS
> 
> Incorrect doxygen API HTML rendering.
> 
>> + */
>> +int
>> +rte_tm_node_stats_update(uint8_t port_id,
>> +    uint32_t node_id,
>> +    uint64_t stats_mask,
>> +    struct rte_tm_error *error);
>> +
>> +/**
>> + * Traffic manager node WFQ weight mode update
>> + *
>> + * @param[in] port_id
>> + *   The port identifier of the Ethernet device.
>> + * @param[in] node_id
>> + *   Node ID. Needs to be valid leaf node ID.
>> + * @param[in] wfq_weight_mode
>> + *   WFQ weight mode for each SP priority. When NULL, it indicates that WFQ is
>> + *   to be used for all priorities. When non-NULL, it points to a pre-allocated
>> + *   array of *n_sp_priorities* values, with non-zero value for byte-mode and
>> + *   zero for packet-mode.
>> + * @param[in] n_sp_priorities
>> + *   Number of SP priorities.
>> + * @param[out] error
>> + *   Error details. Filled in only on error, when not NULL.
>> + * @return
>> + *   0 on success, non-zero error code otherwise.
>> + *
>> + * @see RTE_TM_UPDATE_NODE_WFQ_WEIGHT_MODE
>> + * @see RTE_TM_UPDATE_NODE_N_SP_PRIORITIES
>> + */
>> +int
>> +rte_tm_node_wfq_weight_mode_update(uint8_t port_id,
>> +    uint32_t node_id,
>> +    int *wfq_weight_mode,
>> +    uint32_t n_sp_priorities,
>> +    struct rte_tm_error *error);
>> +
>> +/**
>> + * Traffic manager node congestion management mode update
>> + *
>> + * @param[in] port_id
>> + *   The port identifier of the Ethernet device.
>> + * @param[in] node_id
>> + *   Node ID. Needs to be valid leaf node ID.
>> + * @param[in] cman
>> + *   Congestion management mode.
>> + * @param[out] error
>> + *   Error details. Filled in only on error, when not NULL.
>> + * @return
>> + *   0 on success, non-zero error code otherwise.
>> + *
>> + * @see RTE_TM_UPDATE_NODE_CMAN
>> + */
>> +int
>> +rte_tm_node_cman_update(uint8_t port_id,
>> +    uint32_t node_id,
>> +    enum rte_tm_cman_mode cman,
>> +    struct rte_tm_error *error);
>> +
>> +/**
>> + * Traffic manager node private WRED context update
>> + *
>> + * @param[in] port_id
>> + *   The port identifier of the Ethernet device.
>> + * @param[in] node_id
>> + *   Node ID. Needs to be valid leaf node ID.
>> + * @param[in] wred_profile_id
>> + *   WRED profile ID for the private WRED context of the current node. Needs to
>> + *   be either valid WRED profile ID or RTE_TM_WRED_PROFILE_ID_NONE, with the
>> + *   latter disabling the private WRED context of the current node.
>> + * @param[out] error
>> + *   Error details. Filled in only on error, when not NULL.
>> + * @return
>> + *   0 on success, non-zero error code otherwise.
>> + */
> 
> Missing the @see to point the capability.
> 
>> +int
>> +rte_tm_node_wred_context_update(uint8_t port_id,
>> +    uint32_t node_id,
>> +    uint32_t wred_profile_id,
>> +    struct rte_tm_error *error);
>> +
>> +/**
>> + * Traffic manager node shared WRED context update
>> + *
>> + * @param[in] port_id
>> + *   The port identifier of the Ethernet device.
>> + * @param[in] node_id
>> + *   Node ID. Needs to be valid leaf node ID.
>> + * @param[in] shared_wred_context_id
>> + *   Shared WRED context ID. Needs to be valid.
>> + * @param[in] add
>> + *   Set to non-zero value to add this shared WRED context to current node or
>> + *   to zero to delete this shared WRED context from current node.
>> + * @param[out] error
>> + *   Error details. Filled in only on error, when not NULL.
>> + * @return
>> + *   0 on success, non-zero error code otherwise.
> 
> Missing the @see to point the capability.
> 
>> + * @see struct rte_tm_capabilities::mark_vlan_dei_supported
>> + */
>> +int
>> +rte_tm_mark_vlan_dei(uint8_t port_id,
>> +    int mark_green,
>> +    int mark_yellow,
>> +    int mark_red,
>> +    struct rte_tm_error *error);
>> +
>> +/**
>> + * Traffic manager packet marking - IPv4 / IPv6 ECN (IETF RFC 3168)
>> + *
>> + * IETF RFCs 2474 and 3168 reorganize the IPv4 Type of Service (TOS) field
>> + * (8 bits) and the IPv6 Traffic Class (TC) field (8 bits) into Differentiated
>> + * Services Codepoint (DSCP) field (6 bits) and Explicit Congestion
>> + * Notification (ECN) field (2 bits). The DSCP field is typically used to
>> + * encode the traffic class and/or drop priority (RFC 2597), while the ECN
>> + * field is used by RFC 3168 to implement a congestion notification mechanism
>> + * to be leveraged by transport layer protocols such as TCP and SCTP that have
>> + * congestion control mechanisms.
>> + *
>> + * When congestion is experienced, as alternative to dropping the packet,
>> + * routers can change the ECN field of input packets from 2'b01 or 2'b10
>> + * (values indicating that source endpoint is ECN-capable) to 2'b11 (meaning
>> + * that congestion is experienced). The destination endpoint can use the
>> + * ECN-Echo (ECE) TCP flag to relay the congestion indication back to the
>> + * source endpoint, which acknowledges it back to the destination endpoint with
>> + * the Congestion Window Reduced (CWR) TCP flag.
>> + *
>> + * All IPv4/IPv6 packets of a given color with ECN set to 2’b01 or 2’b10
>> + * carrying TCP or SCTP have their ECN set to 2’b11 if the marking feature is
>> + * enabled for the current color, otherwise the ECN field is left as is.
>> + *
>> + * @param[in] port_id
>> + *   The port identifier of the Ethernet device.
>> + * @param[in] mark_green
>> + *   Set to non-zero value to enable marking of green packets and to zero to
>> + *   disable it.
>> + * @param[in] mark_yellow
>> + *   Set to non-zero value to enable marking of yellow packets and to zero to
>> + *   disable it.
>> + * @param[in] mark_red
>> + *   Set to non-zero value to enable marking of red packets and to zero to
>> + *   disable it.
>> + * @param[out] error
>> + *   Error details. Filled in only on error, when not NULL.
>> + * @return
>> + *   0 on success, non-zero error code otherwise.
>> + *
>> + * @see struct rte_tm_capabilities::mark_ip_ecn_tcp_supported
>> + * @see struct rte_tm_capabilities::mark_ip_ecn_sctp_supported
>> + */
>> +int
>> +rte_tm_mark_ip_ecn(uint8_t port_id,
>> +    int mark_green,
>> +    int mark_yellow,
>> +    int mark_red,
>> +    struct rte_tm_error *error);
>> +
>> +/**
>> + * Traffic manager packet marking - IPv4 / IPv6 DSCP (IETF RFC 2597)
>> + *
>> + * IETF RFC 2597 maps the traffic class and the drop priority to the IPv4/IPv6
>> + * Differentiated Services Codepoint (DSCP) field (6 bits). Here are the DSCP
>> + * values proposed by this RFC:
>> + *
>> + *                       Class 1    Class 2    Class 3    Class 4
>> + *                     +----------+----------+----------+----------+
>> + *    Low Drop Prec    |  001010  |  010010  |  011010  |  100010  |
>> + *    Medium Drop Prec |  001100  |  010100  |  011100  |  100100  |
>> + *    High Drop Prec   |  001110  |  010110  |  011110  |  100110  |
>> + *                     +----------+----------+----------+----------+
> 
> Incorrect doxygen API HTML rendering.