[dpdk-dev] [RFC] Accelerator API to chain packet processing functions

[Coyle, David]

Having taken feedback from the community into account, we would like to propose some changes to our approach for combining multiple packet-processing functions into a single operation on a single device, be that an optimized software library or a hardware accelerator.

The main feedback on the rte_accelerator API can be summarized as follows:
1) Why are we creating another new library that performs tasks similar to other existing APIs... why not try and converge on one?
2) The term "accelerator" is too broad a term, if the API is primarily focused on Crypto + CRC

We also felt that using the rte_cryptodev and rte_compressdev APIs to initialize, configure and reset the devices and then using rte_accelerator for session creation and operation enqueue/dequeue was confusing matters.
We believe the new approach addresses the above concerns and also greatly simplifies the solution.

Our new approach proposes to use the already existing rte_rawdev API with some added functionality for creating "multi-function" sessions.

At the high level, the main changes are:
	- The rte_accelerator library will no longer be added
	- The rte_rawdev API will be used to initialize, configure, reset a device
	- A new rawdev interface for "multi-function" sessions/operations will be added under the new directory 'drivers/raw/common'
		- this interface's header file will be called 'rte_rawdev_multi_fn.h' (with an accompanying C file for function implementations)
		- this header file will contain much of what was previously included in rte_accelerator.h and rte_err_detect.h, such as:
			- enums and structs for defining a multi-function chain of xforms, using xform definitions from rte_cryptodev and rte_compressdev as necessary
			- enums and structs for defining a multi-function chain of ops, again using op definitions from rte_cryptodev and rte_compressdev as necessary
			- enums and structs for defining error-detection xforms and ops
			- two API function definitions to create and destroy a session based on a xform chain
				- rte_rawdev_multi_fn_session_create()
				- rte_rawdev_multi_fn_session_destroy()
		- application code will include rte_rawdev_multi_fn.h to access the structs, enums and functions for creating xform chains, sessions and op chains
		- keeping the multi-function interface under the 'drivers' directory means that rte_rawdev itself remains completely "raw", with no knowledge of xforms, sessions or ops
		- a proposal for this header file is included at the end
	- The rte_rawdev API will be used to enqueue/dequeue the operations using the existing rte_rawdev_enqueue_buffers() and rte_rawdev_dequeue_buffers()
		- a synchronous API function could potentially be added to rte_rawdev in the future if required, to avoid the overhead of enqueue/dequeue for the optimized software library use-case (e.g. rte_rawdev_process_buffers())
	- Two new rawdev PMDs for will be added under 'drivers/raw' for QAT and AESNI-MB
		- these two rawdev PMDs will use and implement the multi-function interface defined in 'drivers/raw/common/rte_rawdev_multi_fn.h'
		- as with all other rawdev PMDs, the interface is known only to the application and the PMD itself, and is opaque to rte_rawdev itself
		- the PMDs will be added under 'drivers/raw/aesni_mb' and 'drivers/raw/qat'
		- other PMDs (existing or new) could use this multi-function interface in the future if use-cases arise
	- The rte_rawdev library will be used as is, with no changes required
	

The initial use cases for the multi-function rawdev interface remain the same as for the previously proposed rte_accelerator:
	- DOCSIS MAC: Crypto + CRC
	- XGS-PON MAC: Crypto + CRC + BIP

However, the API can still also accommodate other chained functions such as Compression + Crypto and UDP Checksum + Crypto.

The following diagram shows the new architecture:

    +-----------------------------------------------------------+
    |                                                           |
    |                      Application                          |
    |        (e.g. vCMTS (DOCSIS), vOLT (XGS-PON), etc.)        |
    |                                                           |
    +-----------------------------------------------------------+
                                |
    +---------------------------|-------------------------------+
    |                           |                       DPDK    |
    |                           |                               |
    |                 +---------------------+                   |
    |                 |                     |                   |
    |                 |     rte_rawdev      |                   |
    |                 |                     |                   |               NOTE:
    |                 +---------------------+       ____________|_______ 'RAWDEV MULTI-FUNCTION
    |                        /      \              /            |           API' is opaque to
    |                       /        \            /             |             rte_rawdev
    |                      /          \          /              |
    |           +--------------------------------+              |
    |           |   RAWDEV MULTI-FUNCTION API    |              |
    |           +--------------------------------+              |
    |           +------------+      +------------+              |
    |           |   RAWDEV   |      |   RAWDEV   |              |
    |           |  AESNI-MB  |      |    QAT     |              |
    |           |    PMD     |      |    PMD     |              |
    |           +------------+      +------------+              |
    |                  |                  |                     |
    +------------------|------------------|---------------------+
                       |                  |
                +------------+      +------------+
                |  AESNI-MB  |      |   QAT HW   |
                |   SW LIB   |      |            |
                +------------+      +------------+


Note that development work is already progressing well on this new approach, with the aim of upstreaming this into DPDK v20.05.

Also, we may consider consolidating the multi-function API into the main DPDK library in the future if there were more devices which wanted to support these multi-function operations.

The following is the proposed rawdev multi-function interface, defined in 'drivers/raw/common/rte_rawdev_multi_fn.h'

/* SPDX-License-Identifier: BSD-3-Clause
 * Copyright(c) 2020 Intel Corporation.
 */

#ifndef _RTE_RAWDEV_MULTI_FN_H_
#define _RTE_RAWDEV_MULTI_FN_H_

#ifdef __cplusplus
extern "C" {
#endif

#include <rte_compat.h>
#include <rte_common.h>
#include <rte_mbuf.h>
#include <rte_memory.h>
#include <rte_mempool.h>
#include <rte_comp.h>
#include <rte_crypto.h>
#include <rte_rawdev.h>

/** Error Detection Algorithms */
enum rte_rawdev_multi_fn_err_detect_algorithm {
	RTE_RAWDEV_MULTI_FN_ERR_DETECT_CRC32_ETH,
	/**< CRC32 Ethernet */
	RTE_RAWDEV_MULTI_FN_ERR_DETECT_BIP32
	/**< BIP32 */
};

/** Error Detection Operation Types */
enum rte_rawdev_multi_fn_err_detect_operation {
	RTE_RAWDEV_MULTI_FN_ERR_DETECT_OP_VERIFY,
	/**< Verify error detection result */
	RTE_RAWDEV_MULTI_FN_ERR_DETECT_OP_GENERATE
	/**< Generate error detection result */
};

/** Error Detection Status */
enum rte_rawdev_multi_fn_err_detect_op_status {
	RTE_RAWDEV_MULTI_FN_ERR_DETECT_OP_STATUS_NOT_PROCESSED,
	/**< Operation has not yet been processed by a device */
	RTE_RAWDEV_MULTI_FN_ERR_DETECT_OP_STATUS_SUCCESS,
	/**< Operation completed successfully */
	RTE_RAWDEV_MULTI_FN_ERR_DETECT_OP_STATUS_VERIFY_FAILED,
	/**< Verification failed */
	RTE_RAWDEV_MULTI_FN_ERR_DETECT_OP_STATUS_ERROR
	/**< Error handling operation */
};

/**
 * Error Detection Transform Data
 *
 * This structure contains data relating to an error detection transform. The
 * fields *op* and *algo* are common to all error detection transforms and
 * MUST be set
 */
struct rte_rawdev_multi_fn_err_detect_xform {
	enum rte_rawdev_multi_fn_err_detect_operation op;
	/**< Error detection operation type */
	enum rte_rawdev_multi_fn_err_detect_algorithm algo;
	/**< Error detection algorithm */
};

/** Error Detection Operation */
struct rte_rawdev_multi_fn_err_detect_op {
	struct rte_mbuf *m_src; /**< Source mbuf */
	enum rte_rawdev_multi_fn_err_detect_op_status status;
	/**< Operation status */

	struct {
		uint16_t offset;
		/**<
		 * Starting point for error detection processing, specified
		 * as the number of bytes from start of the packet in the
		 * source mbuf
		 */
		uint16_t length;
		/**<
		 * The length, in bytes, of the source mbuf on which the error
		 * detection operation will be computed
		 */
	} data; /**< Data offset and length for error detection */

	struct {
		uint8_t *data;
		/**<
		 * This points to the location where the error detection
		 * result should be written (in the case of generation) or
		 * where the purported result exists (in the case of
		 * verification)
		 *
		 * The caller must ensure the required length of physically
		 * contiguous memory is available at this address
		 *
		 * For a CRC, this may point into the mbuf packet data. For
		 * an operation such as a BIP, this may point to a memory
		 * location after the op
		 *
		 * For generation, the result will overwrite any data at this
		 * location
		 */
		rte_iova_t phys_addr;
		/**< Physical address of output data */
	} output; /**< Output location */
};

/**
 * Multi-function transform types
 */
enum rte_rawdev_multi_fn_xform_type {
	RTE_RAWDEV_MULTI_FN_XFORM_TYPE_CRYPTO_SYM,
	/**< Symmetric crypto transform type */
	RTE_RAWDEV_MULTI_FN_XFORM_TYPE_CRYPTO_ASYM,
	/**< Asymmetric crypto transform type */
	RTE_RAWDEV_MULTI_FN_XFORM_TYPE_COMP,
	/**< Compression transform type */
	RTE_RAWDEV_MULTI_FN_XFORM_TYPE_ERR_DETECT
	/**< Error detection transform type */
};

/**
 * Multi-function transform setup data
 *
 * This structure is used to specify the multi-function transforms required.
 * Multiple transforms can be chained together to specify a chain of transforms
 * such as symmetric crypto followed by error detection, or compression followed
 * by symmetric crypto. Each transform structure holds a single transform, with
 * the type field specifying which transform is contained within the union.
 */
struct rte_rawdev_multi_fn_xform {
	struct rte_rawdev_multi_fn_xform *next;
	/**<
	 * Next transform in the chain
	 * - the last transform in the chain MUST set this to NULL
	 */
	enum rte_rawdev_multi_fn_xform_type type;
	/**< Transform type */

	RTE_STD_C11
	union {
		struct rte_crypto_sym_xform crypto_sym;
		/**< Symmetric crypto transform */
		struct rte_crypto_asym_xform crypto_asym;
		/**< Asymmetric crypto transform */
		struct rte_comp_xform comp;
		/**< Compression transform */
		struct rte_rawdev_multi_fn_err_detect_xform err_detect;
		/**< Error detection transform */
	};
};

/**
 * Multi-function operation status
 */
enum rte_rawdev_multi_fn_op_status {
	RTE_RAWDEV_MULTI_FN_OP_STATUS_SUCCESS = 0,
	/**< Operation completed successfully */
	RTE_RAWDEV_MULTI_FN_OP_STATUS_FAILURE,
	/**< Operation completed with failure */
	RTE_RAWDEV_MULTI_FN_STATUS_INVALID_SESSION,
	/**< Operation failed due to invalid session arguments */
	RTE_RAWDEV_MULTI_FN_OP_STATUS_NOT_PROCESSED,
	/**< Operation has not yet been processed by a device */
};

/**
 * Multi-function session data
 */
struct rte_rawdev_multi_fn_session;

/**
 * Multi-function operation data
 *
 * This structure is used to specify the operations for a particular session.
 * This includes specifying the source and, if required, destination mbufs and
 * the lengths and offsets of the data within these mbufs on which the
 * operations should be done. Multiple operations are chained together to
 * specify the full set of operations to be performed
 *
 * @note The rte_rawdev_multi_fn_op chain MUST match the session's xform
 * chain exactly
 * @note The first rte_rawdev_multi_fn_op element in the chain is the parent
 * operation. The following fields MUST be set in this first operation before
 * enqueuing and are ignored in the inner operations and any subsequent
 * rte_rawdev_multi_fn_op chain elements:
 * - *sess*
 * - *m_src*
 * - *m_dst* (if required)
 * @note If *sess* or *m_src* is not set in the first rte_rawdev_multi_fn_op,
 * this operation is invalid and will cause an error when attempting to enqueue.
 * @note The following fields MUST be set in ALL rte_rawdev_multi_fn_op chain
 * elements:
 * - *next*
 * - *mempool*
 * - *type*
 * @note After the operation has been dequeued, only the FIRST (i.e. the parent)
 * rte_rawdev_multi_fn_op in the chain will contain the *overall_status*. Each
 * chain element will contain it's individual *op_status*, the value of which is
 * relevant to operation type (e.g. an ::rte_crypto_op_status,
 * ::rte_comp_op_status or ::rte_err_detect_op_status)
 */
struct rte_rawdev_multi_fn_op {
	struct rte_rawdev_multi_fn_op *next;
	/**<
	 * Next operation in the chain
	 * - the last operation in the chain MUST set this to NULL
	 */
	struct rte_rawdev_multi_fn_session *sess;
	/**< Handle for the associated multi-function session */

	struct rte_mempool *mempool;
	/**< Mempool from which the operation is allocated */

	struct rte_mbuf *m_src; /**< Source mbuf */
	struct rte_mbuf *m_dst; /**< Destination mbuf */

	enum rte_rawdev_multi_fn overall_status;
	/**<
	 * Overall operation status
	 * - indicates if all the operations in the chain succeeded or if any
	 *   one of them failed
	 */

	uint8_t op_status;
	/**<
	 * Individual operation status
	 * - indicates the status of the individual operation in the chain
	 */

	RTE_STD_C11
	union {
		struct rte_crypto_sym_op crypto_sym;
		/**< Symmetric crypto operation */
		struct rte_crypto_asym_op crypto_asym;
		/**< Asymmetric crypto operation */
		struct rte_comp_op comp;
		/**< Compression operation */
		struct rte_rawdev_multi_fn_err_detect_op err_detect;
		/**< Error detection operation */
	};
};

/**
 * Create multi-function session as specified by the transform chain
 *
 * @param   dev_info	Device info, obtained by calling rte_rawdev_info_get()
 * @param   xform	Pointer to the first element of the session transform
 * 			chain
 * @param   socket_id	Socket to allocate the session on
 *
 * @return
 *  - Pointer to session, if successful
 *  - NULL, on failure
 */
__rte_experimental
struct rte_rawdev_multi_fn_session *
rte_rawdev_multi_fn_session_create(struct rte_rawdev_info *dev_info,
				   struct rte_rawdev_multi_fn_xform *xform,
				   int socket_id);

/**
 * Free memory assoicated with a multi-function session
 *
 * @param   dev_info	Device info, obtained by calling rte_rawdev_info_get()
 * @param   sess	Multi-function session to be freed
 *
 * @return
 *  - 0, if successful
 *  - -EINVAL, if session is NULL
 *  - -EBUSY, if not all session data has been freed
 */
__rte_experimental
int
rte_rawdev_multi_fn_session_destroy(struct rte_rawdev_info *dev_info,
				    struct rte_rawdev_multi_fn_session *sess);

#ifdef __cplusplus
}
#endif

#endif /* _RTE_RAWDEV_MULTI_FN_H_ */