[dpdk-dev] [PATCH] crypto/cnxk: fix bus error on RSA verify

[kirankumark at marvell.com]

From: Kiran Kumar K <kirankumark at marvell.com>

While creating RSA session, private key length is not being
calculated properly. This is causing bus error on RSA verify.
This patch fix the issue with length calculation.

Fixes: 5a3513caeb455 ("crypto/cnxk: add asymmetric session")

Signed-off-by: Kiran Kumar K <kirankumark at marvell.com>
---
 drivers/crypto/cnxk/cnxk_ae.h | 4 ++--
 1 file changed, 2 insertions(+), 2 deletions(-)

diff --git a/drivers/crypto/cnxk/cnxk_ae.h b/drivers/crypto/cnxk/cnxk_ae.h
index 00dc75ef9d..6222171fe6 100644
--- a/drivers/crypto/cnxk/cnxk_ae.h
+++ b/drivers/crypto/cnxk/cnxk_ae.h
@@ -82,15 +82,15 @@ cnxk_ae_fill_rsa_params(struct cnxk_ae_sess *sess,
 	struct rte_crypto_rsa_xform *rsa = &sess->rsa_ctx;
 	size_t mod_len = xfrm_rsa->n.length;
 	size_t exp_len = xfrm_rsa->e.length;
-	size_t len = (mod_len / 2);
 	uint64_t total_size;
+	size_t len = 0;
 
 	if (qt.p.length != 0 && qt.p.data == NULL)
 		return -EINVAL;
 
 	/* Make sure key length used is not more than mod_len/2 */
 	if (qt.p.data != NULL)
-		len = RTE_MIN(len, qt.p.length);
+		len = (((mod_len / 2) < qt.p.length) ? 0 : qt.p.length);
 
 	/* Total size required for RSA key params(n,e,(q,dQ,p,dP,qInv)) */
 	total_size = mod_len + exp_len + 5 * len;
-- 
2.25.1