[dpdk-dev] [dpdk-dev v10 2/4] cryptodev: add raw crypto data-path APIs

Akhil Goyal akhil.goyal at nxp.com
Thu Oct 8 16:26:06 CEST 2020


Hi Fan,

> 
> This patch adds raw data-path APIs for enqueue and dequeue
> operations to cryptodev. The APIs support flexible user-define
> enqueue and dequeue behaviors.
> 
> Signed-off-by: Fan Zhang <roy.fan.zhang at intel.com>
> Signed-off-by: Piotr Bronowski <piotrx.bronowski at intel.com>
> ---
>  doc/guides/prog_guide/cryptodev_lib.rst       |  93 +++++
>  doc/guides/rel_notes/release_20_11.rst        |   7 +
>  lib/librte_cryptodev/rte_crypto_sym.h         |   2 +-
>  lib/librte_cryptodev/rte_cryptodev.c          | 104 +++++
>  lib/librte_cryptodev/rte_cryptodev.h          | 354 +++++++++++++++++-
>  lib/librte_cryptodev/rte_cryptodev_pmd.h      |  47 ++-
>  .../rte_cryptodev_version.map                 |  11 +
>  7 files changed, 614 insertions(+), 4 deletions(-)
> 
> diff --git a/doc/guides/prog_guide/cryptodev_lib.rst
> b/doc/guides/prog_guide/cryptodev_lib.rst
> index e7ba35c2d..5fe6c3c24 100644
> --- a/doc/guides/prog_guide/cryptodev_lib.rst
> +++ b/doc/guides/prog_guide/cryptodev_lib.rst
> @@ -632,6 +632,99 @@ a call argument. Status different than zero must be
> treated as error.
>  For more details, e.g. how to convert an mbuf to an SGL, please refer to an
>  example usage in the IPsec library implementation.
> 
> +Cryptodev Raw Data-path APIs
> +~~~~~~~~~~~~~~~~~~~~~~~~~~~~
> +
> +The Crypto Raw data-path APIs are a set of APIs are designed to enable

The Crypto Raw data-path APIs are a set of APIs designed to enable
external libraries/applications to leverage the cryptographic 

> +externel libraries/applications which want to leverage the cryptographic
> +processing provided by DPDK crypto PMDs through the cryptodev API but in a
> +manner that is not dependent on native DPDK data structures (eg. rte_mbuf,
> +rte_crypto_op, ... etc) in their data-path implementation.
> +
> +The raw data-path APIs have the following advantages:
> +- External data structure friendly design. The new APIs uses the operation
> +  descriptor ``struct rte_crypto_sym_vec`` that supports raw data pointer and
> +  IOVA addresses as input. Moreover, the APIs does not require the user to
> +  allocate the descriptor from mempool, nor requiring mbufs to describe input
> +  data's virtual and IOVA addresses. All these features made the translation
> +  from user's own data structure into the descriptor easier and more efficient.
> +- Flexible enqueue and dequeue operation. The raw data-path APIs gives the
> +  user more control to the enqueue and dequeue operations, including the
> +  capability of precious enqueue/dequeue count, abandoning enqueue or
> dequeue
> +  at any time, and operation status translation and set on the fly.
> +
> +Cryptodev PMDs who supports the raw data-path APIs will have

Cryptodev PMDs which support raw data-path APIs will have

> +``RTE_CRYPTODEV_FF_SYM_HW_RAW_DP`` feature flag presented. To use this
> +feature, the user should create a local ``struct rte_crypto_raw_dp_ctx``
> +buffer and extend to at least the length returned by
> +``rte_cryptodev_raw_get_dp_context_size`` function call. The created buffer

``rte_cryptodev_get_raw_dp_ctx_size``

> +is then configured using ``rte_cryptodev_raw_configure_dp_context`` function.
rte_cryptodev _configure_raw_dp_ctx

> +The library and the crypto device driver will then configure the buffer and
> +write necessary temporary data into the buffer for later enqueue and dequeue
> +operations. The temporary data may be treated as the shadow copy of the
> +driver's private queue pair data.
> +
> +After the ``struct rte_crypto_raw_dp_ctx`` buffer is initialized, it is then
> +attached either the cryptodev sym session, the rte_security session, or the

attached either with the cryptodev sym session, the rte_security session, or the

> +cryptodev xform for session-less operation by
> +``rte_cryptodev_raw_attach_session`` function. With the session or xform

``rte_cryptodev_attach_raw_session`` API

> +information the driver will set the corresponding enqueue and dequeue
> function
> +handlers to the ``struct rte_crypto_raw_dp_ctx`` buffer.
> +
> +After the session is attached, the ``struct rte_crypto_raw_dp_ctx`` buffer is
> +now ready for enqueue and dequeue operation. There are two different
> enqueue
> +functions: ``rte_cryptodev_raw_enqueue`` to enqueue single descriptor,
> +and ``rte_cryptodev_raw_enqueue_burst`` to enqueue multiple descriptors.
> +In case of the application uses similar approach to
> +``struct rte_crypto_sym_vec`` to manage its data burst but with different
> +data structure, using the ``rte_cryptodev_raw_enqueue_burst`` function may
> be
> +less efficient as this is a situation where the application has to loop over
> +all crypto descriptors to assemble the ``struct rte_crypto_sym_vec`` buffer
> +from its own data structure, and then the driver will loop over them again to
> +translate every crypto job to the driver's specific queue data. The
> +``rte_cryptodev_raw_enqueue`` should be used to save one loop for each data
> +burst instead.
> +
> +During the enqueue, the cryptodev driver only sets the enqueued descriptors
> +into the device queue but not initiates the device to start processing them.
> +The temporary queue pair data changes in relation to the enqueued descriptors
> +may be recorded in the ``struct rte_crypto_raw_dp_ctx`` buffer as the
> reference
> +to the next enqueue function call. When ``rte_cryptodev_raw_enqueue_done``
> is
> +called, the driver will initiate the processing of all enqueued descriptors and
> +merge the temporary queue pair data changes into the driver's private queue
> +pair data. Calling ``rte_cryptodev_raw_configure_dp_context`` twice without
> +``rte_cryptodev_dp_enqueue_done`` call in between will invalidate the
> temporary
> +data stored in ``struct rte_crypto_raw_dp_ctx`` buffer. This feature is useful
> +when the user wants to abandon partially enqueued data for a failed enqueue
> +burst operation and try enqueuing in a whole later.

This feature may not be supported by all the HW PMDs, Can there be a way to bypass
this done API?

> +
> +Similar as enqueue, there are two dequeue functions:
> +``rte_cryptodev_raw_dequeue`` for dequeing single descriptor, and
> +``rte_cryptodev_raw_dequeue_burst`` for dequeuing a burst of descriptor. The
> +dequeue functions only writes back the user data that was passed to the driver
> +during inqueue, and inform the application the operation status.

during enqueue, and inform the operation status to the application.

> +Different than ``rte_cryptodev_dequeue_burst`` which the user can only
> +set an expected dequeue count and needs to read from dequeued cryptodev
> +operations' status field, the raw data-path dequeue burst function allows
> +the user to provide callback functions to retrieve dequeue
> +count from the enqueued user data, and write the expected status value to the
> +user data on the fly.
> +
> +Same as enqueue, both ``rte_cryptodev_raw_dequeue`` and
> +``rte_cryptodev_raw_dequeue_burst`` will not wipe the dequeued descriptors
> +from cryptodev queue unless ``rte_cryptodev_dp_dequeue_done`` is called.
> The
> +dequeue related temporary queue data will be merged into the driver's private
> +queue data in the function call.
> +
> +There are a few limitations to the data path service:
> +
> +* Only support in-place operations.
> +* APIs are NOT thread-safe.
> +* CANNOT mix the direct API's enqueue with rte_cryptodev_enqueue_burst, or
> +  vice versa.
> +
> +See *DPDK API Reference* for details on each API definitions.
> +
>  Sample code
>  -----------
> 
> diff --git a/doc/guides/rel_notes/release_20_11.rst
> b/doc/guides/rel_notes/release_20_11.rst
> index 20ebaef5b..d3d9f82f7 100644
> --- a/doc/guides/rel_notes/release_20_11.rst
> +++ b/doc/guides/rel_notes/release_20_11.rst
> @@ -55,6 +55,13 @@ New Features
>       Also, make sure to start the actual text at the margin.
>       =======================================================
> 
> +   * **Added raw data-path APIs for cryptodev library.**
> +
> +     Cryptodev is added raw data-path APIs to accelerate external libraries or
> +     applications those want to avail fast cryptodev enqueue/dequeue
> +     operations but does not necessarily depends on mbufs and cryptodev
> +     operation mempool.

Raw crypto data path APIs are added to accelerate external libraries or
Applications which need to perform crypto processing on raw buffers and
Not dependent on rte_mbufs or rte_crypto_op mempools.

> +
> 
>  Removed Items
>  -------------
> diff --git a/lib/librte_cryptodev/rte_crypto_sym.h
> b/lib/librte_cryptodev/rte_crypto_sym.h
> index 8201189e0..e1f23d303 100644
> --- a/lib/librte_cryptodev/rte_crypto_sym.h
> +++ b/lib/librte_cryptodev/rte_crypto_sym.h
> @@ -57,7 +57,7 @@ struct rte_crypto_sgl {
>   */
>  struct rte_crypto_va_iova_ptr {
>  	void *va;
> -	rte_iova_t *iova;
> +	rte_iova_t iova;
>  };

This should be part of 1/4 of this patchset.

> 
>  /**
> diff --git a/lib/librte_cryptodev/rte_cryptodev.c
> b/lib/librte_cryptodev/rte_cryptodev.c
> index 1dd795bcb..daeb5f504 100644
> --- a/lib/librte_cryptodev/rte_cryptodev.c
> +++ b/lib/librte_cryptodev/rte_cryptodev.c
> @@ -1914,6 +1914,110 @@ rte_cryptodev_sym_cpu_crypto_process(uint8_t
> dev_id,
>  	return dev->dev_ops->sym_cpu_process(dev, sess, ofs, vec);
>  }
> 
> +int
> +rte_cryptodev_raw_get_dp_context_size(uint8_t dev_id)

As suggested above raw_dp should be used as keyword in all the APIs
Hence it should be rte_cryptodev_get_raw_dp_ctx_size

> +{
> +	struct rte_cryptodev *dev;
> +	int32_t size = sizeof(struct rte_crypto_raw_dp_ctx);
> +	int32_t priv_size;
> +
> +	if (!rte_cryptodev_pmd_is_valid_dev(dev_id))
> +		return -EINVAL;
> +
> +	dev = rte_cryptodev_pmd_get_dev(dev_id);
> +
> +	if (*dev->dev_ops->get_drv_ctx_size == NULL ||
> +		!(dev->feature_flags &
> RTE_CRYPTODEV_FF_SYM_HW_RAW_DP)) {
> +		return -ENOTSUP;
> +	}
> +
> +	priv_size = (*dev->dev_ops->get_drv_ctx_size)(dev);
> +	if (priv_size < 0)
> +		return -ENOTSUP;
> +
> +	return RTE_ALIGN_CEIL((size + priv_size), 8);
> +}
> +
> +int
> +rte_cryptodev_raw_configure_dp_context(uint8_t dev_id, uint16_t qp_id,
> +	struct rte_crypto_raw_dp_ctx *ctx)

rte_cryptodev_configure_raw_dp_ctx

> +{
> +	struct rte_cryptodev *dev;
> +	union rte_cryptodev_session_ctx sess_ctx = {NULL};
> +
> +	if (!rte_cryptodev_get_qp_status(dev_id, qp_id))
> +		return -EINVAL;
> +
> +	dev = rte_cryptodev_pmd_get_dev(dev_id);
> +	if (!(dev->feature_flags & RTE_CRYPTODEV_FF_SYM_HW_RAW_DP)
> +			|| dev->dev_ops->configure_dp_ctx == NULL)
> +		return -ENOTSUP;
> +
> +	return (*dev->dev_ops->configure_dp_ctx)(dev, qp_id,
> +			RTE_CRYPTO_OP_WITH_SESSION, sess_ctx, ctx);
> +}
> +
> +int
> +rte_cryptodev_raw_attach_session(uint8_t dev_id, uint16_t qp_id,
> +	struct rte_crypto_raw_dp_ctx *ctx,
> +	enum rte_crypto_op_sess_type sess_type,
> +	union rte_cryptodev_session_ctx session_ctx)
> +{
> +	struct rte_cryptodev *dev;
> +
> +	if (!rte_cryptodev_get_qp_status(dev_id, qp_id))
> +		return -EINVAL;
> +
> +	dev = rte_cryptodev_pmd_get_dev(dev_id);
> +	if (!(dev->feature_flags & RTE_CRYPTODEV_FF_SYM_HW_RAW_DP)
> +			|| dev->dev_ops->configure_dp_ctx == NULL)
> +		return -ENOTSUP;
> +	return (*dev->dev_ops->configure_dp_ctx)(dev, qp_id, sess_type,
> +			session_ctx, ctx);

What is the difference between rte_cryptodev_raw_configure_dp_context and
rte_cryptodev_raw_attach_session?
And if at all it is needed, then it should be rte_cryptodev_attach_raw_dp_session.
IMO attach is not needed, I am not clear about it.

You are calling the same dev_ops for both - one with explicit session time and other
>From an argument.

> +}
> +
> +uint32_t
> +rte_cryptodev_raw_enqueue_burst(struct rte_crypto_raw_dp_ctx *ctx,
> +	struct rte_crypto_sym_vec *vec, union rte_crypto_sym_ofs ofs,
> +	void **user_data)
> +{
> +	if (vec->num == 1) {

Why do we need this check? I think user is aware that for enqueuing 1 vector,
He should use the other API. Driver will be doing the enqueue operation only one time.

> +		vec->status[0] = rte_cryptodev_raw_enqueue(ctx, vec->sgl-
> >vec,
> +			vec->sgl->num, ofs, vec->iv, vec->digest, vec->aad,
> +			user_data[0]);
> +		return (vec->status[0] == 0) ? 1 : 0;
> +	}
> +
> +	return (*ctx->enqueue_burst)(ctx->qp_data, ctx->drv_ctx_data, vec,
> +			ofs, user_data);
> +}

Where are  rte_cryptodev_raw_enqueue and rte_cryptodev_raw_dequeue ??


> +
> +int
> +rte_cryptodev_raw_enqueue_done(struct rte_crypto_raw_dp_ctx *ctx,
> +		uint32_t n)
> +{
> +	return (*ctx->enqueue_done)(ctx->qp_data, ctx->drv_ctx_data, n);
> +}
> +
> +int
> +rte_cryptodev_raw_dequeue_done(struct rte_crypto_raw_dp_ctx *ctx,
> +		uint32_t n)
> +{
> +	return (*ctx->dequeue_done)(ctx->qp_data, ctx->drv_ctx_data, n);
> +}
> +
> +uint32_t
> +rte_cryptodev_raw_dequeue_burst(struct rte_crypto_raw_dp_ctx *ctx,
> +	rte_cryptodev_raw_get_dequeue_count_t get_dequeue_count,
> +	rte_cryptodev_raw_post_dequeue_t post_dequeue,
> +	void **out_user_data, uint8_t is_user_data_array,
> +	uint32_t *n_success_jobs)
> +{
> +	return (*ctx->dequeue_burst)(ctx->qp_data, ctx->drv_ctx_data,
> +		get_dequeue_count, post_dequeue, out_user_data,
> +		is_user_data_array, n_success_jobs);
> +}
> +
>  /** Initialise rte_crypto_op mempool element */
>  static void
>  rte_crypto_op_init(struct rte_mempool *mempool,
> diff --git a/lib/librte_cryptodev/rte_cryptodev.h
> b/lib/librte_cryptodev/rte_cryptodev.h
> index 7b3ebc20f..3579ab66e 100644
> --- a/lib/librte_cryptodev/rte_cryptodev.h
> +++ b/lib/librte_cryptodev/rte_cryptodev.h
> @@ -466,7 +466,8 @@ rte_cryptodev_asym_get_xform_enum(enum
> rte_crypto_asym_xform_type *xform_enum,
>  /**< Support symmetric session-less operations */
>  #define RTE_CRYPTODEV_FF_NON_BYTE_ALIGNED_DATA		(1ULL
> << 23)
>  /**< Support operations on data which is not byte aligned */
> -
> +#define RTE_CRYPTODEV_FF_SYM_HW_RAW_DP			(1ULL
> << 24)

RTE_CRYPTODEV_FF_SYM_RAW_DP should be better

Add this in doc/guides/cryptodevs/features/default.ini as well in this patch.

> +/**< Support accelerated specific raw data-path APIs */
> 
>  /**
>   * Get the name of a crypto device feature flag
> @@ -1351,6 +1352,357 @@ rte_cryptodev_sym_cpu_crypto_process(uint8_t
> dev_id,
>  	struct rte_cryptodev_sym_session *sess, union rte_crypto_sym_ofs ofs,
>  	struct rte_crypto_sym_vec *vec);
> 
> +/**
> + * Get the size of the raw data-path context buffer.
> + *
> + * @param	dev_id		The device identifier.
> + *
> + * @return
> + *   - If the device supports raw data-path APIs, return the context size.
> + *   - If the device does not support the APIs, return -1.
> + */
> +__rte_experimental
> +int
> +rte_cryptodev_raw_get_dp_context_size(uint8_t dev_id);
> +
> +/**
> + * Union of different crypto session types, including session-less xform
> + * pointer.
> + */
> +union rte_cryptodev_session_ctx {
> +	struct rte_cryptodev_sym_session *crypto_sess;
> +	struct rte_crypto_sym_xform *xform;
> +	struct rte_security_session *sec_sess;
> +};
> +
> +/**
> + * Enqueue a data vector into device queue but the driver will not start
> + * processing until rte_cryptodev_raw_enqueue_done() is called.
> + *
> + * @param	qp		Driver specific queue pair data.
> + * @param	drv_ctx		Driver specific context data.
> + * @param	vec		The array of descriptor vectors.
> + * @param	ofs		Start and stop offsets for auth and cipher
> + *				operations.
> + * @param	user_data	The array of user data for dequeue later.
> + * @return
> + *   - The number of descriptors successfully submitted.
> + */
> +typedef uint32_t (*cryptodev_dp_sym_enqueue_burst_t)(
> +	void *qp, uint8_t *drv_ctx, struct rte_crypto_sym_vec *vec,
> +	union rte_crypto_sym_ofs ofs, void *user_data[]);
> +
> +/**
> + * Enqueue single descriptor into device queue but the driver will not start
> + * processing until rte_cryptodev_raw_enqueue_done() is called.
> + *
> + * @param	qp		Driver specific queue pair data.
> + * @param	drv_ctx		Driver specific context data.
> + * @param	data_vec	The buffer data vector.
> + * @param	n_data_vecs	Number of buffer data vectors.
> + * @param	ofs		Start and stop offsets for auth and cipher
> + *				operations.
> + * @param	iv		IV virtual and IOVA addresses
> + * @param	digest		digest virtual and IOVA addresses
> + * @param	aad_or_auth_iv	AAD or auth IV virtual and IOVA addresses,
> + *				depends on the algorithm used.
> + * @param	user_data	The user data.
> + * @return
> + *   - On success return 0.
> + *   - On failure return negative integer.
> + */
> +typedef int (*cryptodev_dp_sym_enqueue_t)(
> +	void *qp, uint8_t *drv_ctx, struct rte_crypto_vec *data_vec,
> +	uint16_t n_data_vecs, union rte_crypto_sym_ofs ofs,
> +	struct rte_crypto_va_iova_ptr *iv,
> +	struct rte_crypto_va_iova_ptr *digest,
> +	struct rte_crypto_va_iova_ptr *aad_or_auth_iv,
> +	void *user_data);
> +
> +/**
> + * Inform the cryptodev queue pair to start processing or finish dequeuing all
> + * enqueued/dequeued descriptors.
> + *
> + * @param	qp		Driver specific queue pair data.
> + * @param	drv_ctx		Driver specific context data.
> + * @param	n		The total number of processed descriptors.
> + * @return
> + *   - On success return 0.
> + *   - On failure return negative integer.
> + */
> +typedef int (*cryptodev_dp_sym_operation_done_t)(void *qp, uint8_t *drv_ctx,
> +	uint32_t n);
> +
> +/**
> + * Typedef that the user provided for the driver to get the dequeue count.
> + * The function may return a fixed number or the number parsed from the user
> + * data stored in the first processed descriptor.
> + *
> + * @param	user_data	Dequeued user data.
> + **/
> +typedef uint32_t (*rte_cryptodev_raw_get_dequeue_count_t)(void
> *user_data);
> +
> +/**
> + * Typedef that the user provided to deal with post dequeue operation, such
> + * as filling status.
> + *
> + * @param	user_data	Dequeued user data.
> + * @param	index		Index number of the processed descriptor.
> + * @param	is_op_success	Operation status provided by the driver.
> + **/
> +typedef void (*rte_cryptodev_raw_post_dequeue_t)(void *user_data,
> +	uint32_t index, uint8_t is_op_success);
> +
> +/**
> + * Dequeue symmetric crypto processing of user provided data.
> + *
> + * @param	qp			Driver specific queue pair data.
> + * @param	drv_ctx			Driver specific context data.
> + * @param	get_dequeue_count	User provided callback function to
> + *					obtain dequeue count.
> + * @param	post_dequeue		User provided callback function to
> + *					post-process a dequeued operation.
> + * @param	out_user_data		User data pointer array to be retrieve
> + *					from device queue. In case of
> + *					*is_user_data_array* is set there
> + *					should be enough room to store all
> + *					user data.
> + * @param	is_user_data_array	Set 1 if every dequeued user data will
> + *					be written into out_user_data* array.
> + * @param	n_success		Driver written value to specific the
> + *					total successful operations count.
> + *
> + * @return
> + *  - Returns number of dequeued packets.
> + */
> +typedef uint32_t (*cryptodev_dp_sym_dequeue_burst_t)(void *qp,
> +	uint8_t *drv_ctx,
> +	rte_cryptodev_raw_get_dequeue_count_t get_dequeue_count,
> +	rte_cryptodev_raw_post_dequeue_t post_dequeue,
> +	void **out_user_data, uint8_t is_user_data_array,
> +	uint32_t *n_success);
> +
> +/**
> + * Dequeue symmetric crypto processing of user provided data.
> + *
> + * @param	qp			Driver specific queue pair data.
> + * @param	drv_ctx			Driver specific context data.
> + * @param	out_user_data		User data pointer to be retrieve from
> + *					device queue.
> + *
> + * @return
> + *   - 1 if the user_data is dequeued and the operation is a success.
> + *   - 0 if the user_data is dequeued but the operation is failed.
> + *   - -1 if no operation is dequeued.
> + */
> +typedef int (*cryptodev_dp_sym_dequeue_t)(
> +		void *qp, uint8_t *drv_ctx, void **out_user_data);
> +
> +/**
> + * Context data for raw data-path API crypto process. The buffer of this
> + * structure is to be allocated by the user application with the size equal
> + * or bigger than rte_cryptodev_raw_get_dp_context_size() returned value.
> + *
> + * NOTE: the buffer is to be used and maintained by the cryptodev driver, the
> + * user should NOT alter the buffer content to avoid application or system
> + * crash.
> + */
> +struct rte_crypto_raw_dp_ctx {
> +	void *qp_data;
> +
> +	cryptodev_dp_sym_enqueue_t enqueue;
> +	cryptodev_dp_sym_enqueue_burst_t enqueue_burst;
> +	cryptodev_dp_sym_operation_done_t enqueue_done;
> +	cryptodev_dp_sym_dequeue_t dequeue;
> +	cryptodev_dp_sym_dequeue_burst_t dequeue_burst;
> +	cryptodev_dp_sym_operation_done_t dequeue_done;

These function pointers are data path only. Why do we need to add explicit dp in each one of them
These should be cryptodev_sym_raw_**

> +
> +	/* Driver specific context data */
> +	__extension__ uint8_t drv_ctx_data[];
> +};
> +
> +/**
> + * Configure raw data-path context data.
> + *
> + * NOTE:
> + * After the context data is configured, the user should call
> + * rte_cryptodev_raw_attach_session() before using it in
> + * rte_cryptodev_raw_enqueue/dequeue function call.

I am not clear of the purpose of attach API? It looks an overhead to me.

> + *
> + * @param	dev_id		The device identifier.
> + * @param	qp_id		The index of the queue pair from which to
> + *				retrieve processed packets. The value must be
> + *				in the range [0, nb_queue_pair - 1] previously
> + *				supplied to rte_cryptodev_configure().
> + * @param	ctx		The raw data-path context data.
> + * @return
> + *   - On success return 0.
> + *   - On failure return negative integer.
> + */
> +__rte_experimental
> +int
> +rte_cryptodev_raw_configure_dp_context(uint8_t dev_id, uint16_t qp_id,
> +	struct rte_crypto_raw_dp_ctx *ctx);
> +
> +/**
> + * Attach a cryptodev session to a initialized raw data path context.
> + *
> + * @param	dev_id		The device identifier.
> + * @param	qp_id		The index of the queue pair from which to
> + *				retrieve processed packets. The value must be
> + *				in the range [0, nb_queue_pair - 1] previously
> + *				supplied to rte_cryptodev_configure().
> + * @param	ctx		The raw data-path context data.
> + * @param	sess_type	session type.
> + * @param	session_ctx	Session context data.
> + * @return
> + *   - On success return 0.
> + *   - On failure return negative integer.
> + */
> +__rte_experimental
> +int
> +rte_cryptodev_raw_attach_session(uint8_t dev_id, uint16_t qp_id,
> +	struct rte_crypto_raw_dp_ctx *ctx,
> +	enum rte_crypto_op_sess_type sess_type,
> +	union rte_cryptodev_session_ctx session_ctx);
> +
> +/**
> + * Enqueue single raw data-path descriptor.
> + *
> + * The enqueued descriptor will not be started processing until
> + * rte_cryptodev_raw_enqueue_done() is called.
> + *
> + * @param	ctx		The initialized raw data-path context data.
> + * @param	data_vec	The buffer vector.
> + * @param	n_data_vecs	Number of buffer vectors.
> + * @param	ofs		Start and stop offsets for auth and cipher
> + *				operations.
> + * @param	iv		IV virtual and IOVA addresses
> + * @param	digest		digest virtual and IOVA addresses
> + * @param	aad_or_auth_iv	AAD or auth IV virtual and IOVA addresses,
> + *				depends on the algorithm used.
> + * @param	user_data	The user data.
> + * @return
> + *   - The number of descriptors successfully enqueued.
> + */
> +__rte_experimental
> +static __rte_always_inline int
> +rte_cryptodev_raw_enqueue(struct rte_crypto_raw_dp_ctx *ctx,
> +	struct rte_crypto_vec *data_vec, uint16_t n_data_vecs,
> +	union rte_crypto_sym_ofs ofs,
> +	struct rte_crypto_va_iova_ptr *iv,
> +	struct rte_crypto_va_iova_ptr *digest,
> +	struct rte_crypto_va_iova_ptr *aad_or_auth_iv,
> +	void *user_data)
> +{
> +	return (*ctx->enqueue)(ctx->qp_data, ctx->drv_ctx_data, data_vec,
> +		n_data_vecs, ofs, iv, digest, aad_or_auth_iv, user_data);
> +}
> +
> +/**
> + * Enqueue a data vector of raw data-path descriptors.
> + *
> + * The enqueued descriptors will not be started processing until
> + * rte_cryptodev_raw_enqueue_done() is called.
> + *
> + * @param	ctx		The initialized raw data-path context data.
> + * @param	vec		The array of descriptor vectors.
> + * @param	ofs		Start and stop offsets for auth and cipher
> + *				operations.
> + * @param	user_data	The array of opaque data for dequeue.
> + * @return
> + *   - The number of descriptors successfully enqueued.
> + */
> +__rte_experimental
> +uint32_t
> +rte_cryptodev_raw_enqueue_burst(struct rte_crypto_raw_dp_ctx *ctx,
> +	struct rte_crypto_sym_vec *vec, union rte_crypto_sym_ofs ofs,
> +	void **user_data);
> +
> +/**
> + * Start processing all enqueued descriptors from last
> + * rte_cryptodev_raw_configure_dp_context() call.
> + *
> + * @param	ctx	The initialized raw data-path context data.
> + * @param	n	The total number of submitted descriptors.
> + */
> +__rte_experimental
> +int
> +rte_cryptodev_raw_enqueue_done(struct rte_crypto_raw_dp_ctx *ctx,
> +		uint32_t n);
> +
> +/**
> + * Dequeue a burst of raw crypto data-path operations and write the previously
> + * enqueued user data into the array provided.
> + *
> + * The dequeued operations, including the user data stored, will not be
> + * wiped out from the device queue until rte_cryptodev_raw_dequeue_done()
> is
> + * called.
> + *
> + * @param	ctx			The initialized raw data-path context
> + *					data.
> + * @param	get_dequeue_count	User provided callback function to
> + *					obtain dequeue count.
> + * @param	post_dequeue		User provided callback function to
> + *					post-process a dequeued operation.
> + * @param	out_user_data		User data pointer array to be retrieve
> + *					from device queue. In case of
> + *					*is_user_data_array* is set there
> + *					should be enough room to store all
> + *					user data.
> + * @param	is_user_data_array	Set 1 if every dequeued user data will
> + *					be written into *out_user_data* array.
> + * @param	n_success		Driver written value to specific the
> + *					total successful operations count.

// to specify the

> + *
> + * @return
> + *   - Returns number of dequeued packets.
> + */
> +__rte_experimental
> +uint32_t
> +rte_cryptodev_raw_dequeue_burst(struct rte_crypto_raw_dp_ctx *ctx,
> +	rte_cryptodev_raw_get_dequeue_count_t get_dequeue_count,
> +	rte_cryptodev_raw_post_dequeue_t post_dequeue,
> +	void **out_user_data, uint8_t is_user_data_array,
> +	uint32_t *n_success);
> +
> +/**
> + * Dequeue a raw crypto data-path operation and write the previously
> + * enqueued user data.
> + *
> + * The dequeued operation, including the user data stored, will not be wiped
> + * out from the device queue until rte_cryptodev_raw_dequeue_done() is
> called.
> + *
> + * @param	ctx			The initialized raw data-path context
> + *					data.
> + * @param	out_user_data		User data pointer to be retrieve from
> + *					device queue. The driver shall support
> + *					NULL input of this parameter.
> + *
> + * @return
> + *   - 1 if the user data is dequeued and the operation is a success.
> + *   - 0 if the user data is dequeued but the operation is failed.
> + *   - -1 if no operation is ready to be dequeued.
> + */
> +__rte_experimental
> +static __rte_always_inline int

Why is this function specifically inline and not others?

> +rte_cryptodev_raw_dequeue(struct rte_crypto_raw_dp_ctx *ctx,
> +	void **out_user_data)
> +{
> +	return (*ctx->dequeue)(ctx->qp_data, ctx->drv_ctx_data,
> out_user_data);
> +}
> +
> +/**
> + * Inform the queue pair dequeue operations finished.
> + *
> + * @param	ctx	The initialized raw data-path context data.
> + * @param	n	The total number of jobs already dequeued.
> + */
> +__rte_experimental
> +int
> +rte_cryptodev_raw_dequeue_done(struct rte_crypto_raw_dp_ctx *ctx,
> +		uint32_t n);
> +
>  #ifdef __cplusplus
>  }
>  #endif
> diff --git a/lib/librte_cryptodev/rte_cryptodev_pmd.h
> b/lib/librte_cryptodev/rte_cryptodev_pmd.h
> index 81975d72b..69a2a6d64 100644
> --- a/lib/librte_cryptodev/rte_cryptodev_pmd.h
> +++ b/lib/librte_cryptodev/rte_cryptodev_pmd.h
> @@ -316,6 +316,40 @@ typedef uint32_t
> (*cryptodev_sym_cpu_crypto_process_t)
>  	(struct rte_cryptodev *dev, struct rte_cryptodev_sym_session *sess,
>  	union rte_crypto_sym_ofs ofs, struct rte_crypto_sym_vec *vec);
> 
> +/**
> + * Typedef that the driver provided to get service context private date size.
> + *
> + * @param	dev	Crypto device pointer.
> + *
> + * @return
> + *   - On success return the size of the device's service context private data.
> + *   - On failure return negative integer.
> + */
> +typedef int (*cryptodev_dp_get_service_ctx_size_t)(
> +	struct rte_cryptodev *dev);
> +
> +/**
> + * Typedef that the driver provided to configure data-path context.
> + *
> + * @param	dev		Crypto device pointer.
> + * @param	qp_id		Crypto device queue pair index.
> + * @param	service_type	Type of the service requested.
> + * @param	sess_type	session type.
> + * @param	session_ctx	Session context data. If NULL the driver
> + *				shall only configure the drv_ctx_data in
> + *				ctx buffer. Otherwise the driver shall only
> + *				parse the session_ctx to set appropriate
> + *				function pointers in ctx.
> + * @param	ctx		The raw data-path context data.
> + * @return
> + *   - On success return 0.
> + *   - On failure return negative integer.
> + */
> +typedef int (*cryptodev_dp_configure_ctx_t)(
> +	struct rte_cryptodev *dev, uint16_t qp_id,
> +	enum rte_crypto_op_sess_type sess_type,
> +	union rte_cryptodev_session_ctx session_ctx,
> +	struct rte_crypto_raw_dp_ctx *ctx);

These typedefs names are not matching with the corresponding API names.
Can you fix it for all of them?

> 
>  /** Crypto device operations function pointer table */
>  struct rte_cryptodev_ops {
> @@ -348,8 +382,17 @@ struct rte_cryptodev_ops {
>  	/**< Clear a Crypto sessions private data. */
>  	cryptodev_asym_free_session_t asym_session_clear;
>  	/**< Clear a Crypto sessions private data. */
> -	cryptodev_sym_cpu_crypto_process_t sym_cpu_process;
> -	/**< process input data synchronously (cpu-crypto). */
> +	union {
> +		cryptodev_sym_cpu_crypto_process_t sym_cpu_process;
> +		/**< process input data synchronously (cpu-crypto). */
> +		__extension__
> +		struct {
> +			cryptodev_dp_get_service_ctx_size_t get_drv_ctx_size;
> +			/**< Get data path service context data size. */
> +			cryptodev_dp_configure_ctx_t configure_dp_ctx;
> +			/**< Initialize crypto service ctx data. */
> +		};
> +	};
>  };
> 
> 
> diff --git a/lib/librte_cryptodev/rte_cryptodev_version.map
> b/lib/librte_cryptodev/rte_cryptodev_version.map
> index 02f6dcf72..bc4cd1ea5 100644
> --- a/lib/librte_cryptodev/rte_cryptodev_version.map
> +++ b/lib/librte_cryptodev/rte_cryptodev_version.map
> @@ -105,4 +105,15 @@ EXPERIMENTAL {
> 
>  	# added in 20.08
>  	rte_cryptodev_get_qp_status;
> +
> +	# added in 20.11
> +	rte_cryptodev_raw_attach_session;
> +	rte_cryptodev_raw_configure_dp_context;
> +	rte_cryptodev_raw_get_dp_context_size;
> +	rte_cryptodev_raw_dequeue;
> +	rte_cryptodev_raw_dequeue_burst;
> +	rte_cryptodev_raw_dequeue_done;
> +	rte_cryptodev_raw_enqueue;
> +	rte_cryptodev_raw_enqueue_burst;
> +	rte_cryptodev_raw_enqueue_done;
>  };
> --
> 2.20.1



More information about the dev mailing list