[dpdk-stable] patch 'vdpa/mlx5: retry VAR allocation during vDPA restart' has been queued to stable release 20.11.4

[Xueming Li]

Hi,

FYI, your patch has been queued to stable release 20.11.4

Note it hasn't been pushed to http://dpdk.org/browse/dpdk-stable yet.
It will be pushed if I get no objections before 11/12/21. So please
shout if anyone has objections.

Also note that after the patch there's a diff of the upstream commit vs the
patch applied to the branch. This will indicate if there was any rebasing
needed to apply to the stable branch. If there were code changes for rebasing
(ie: not only metadata diffs), please double check that the rebase was
correctly done.

Queued patches are on a temporary branch at:
https://github.com/steevenlee/dpdk

This queued commit can be viewed at:
https://github.com/steevenlee/dpdk/commit/5497fdbe2b4a9b452b42e7805b0ef7d8395ae86b

Thanks.

Xueming Li <xuemingl at nvidia.com>

---
>From 5497fdbe2b4a9b452b42e7805b0ef7d8395ae86b Mon Sep 17 00:00:00 2001
From: Xueming Li <xuemingl at nvidia.com>
Date: Fri, 15 Oct 2021 23:05:45 +0800
Subject: [PATCH] vdpa/mlx5: retry VAR allocation during vDPA restart
Cc: Xueming Li <xuemingl at nvidia.com>

[ upstream commit 8011a09adda4d99121bcc0fe34c98356559eb32f ]

VAR is the device memory space for the virtio queues doorbells,
Qemu could mmap it to directly to speed up doorbell push.

On a busy system, Qemu takes time to release VAR resources during driver
shutdown. If vdpa restarted quickly, the VAR allocation failed with
error 28 since the VAR is singleton resource per device.

This patch adds retry mechanism for VAR allocation.

Fixes: 4cae722c1b06 ("vdpa/mlx5: move virtual doorbell alloc to probe")

Signed-off-by: Xueming Li <xuemingl at nvidia.com>
Reviewed-by: Matan Azrad <matan at nvidia.com>
Reviewed-by: Maxime Coquelin <maxime.coquelin at redhat.com>
---
 drivers/vdpa/mlx5/mlx5_vdpa.c | 10 +++++++++-
 1 file changed, 9 insertions(+), 1 deletion(-)

diff --git a/drivers/vdpa/mlx5/mlx5_vdpa.c b/drivers/vdpa/mlx5/mlx5_vdpa.c
index 6029cfc3a8..38a93d041d 100644
--- a/drivers/vdpa/mlx5/mlx5_vdpa.c
+++ b/drivers/vdpa/mlx5/mlx5_vdpa.c
@@ -686,6 +686,7 @@ mlx5_vdpa_pci_probe(struct rte_pci_driver *pci_drv __rte_unused,
 	struct mlx5_vdpa_priv *priv = NULL;
 	struct ibv_context *ctx = NULL;
 	struct mlx5_hca_attr attr;
+	int retry;
 	int ret;
 
 	ibv = mlx5_vdpa_get_ib_device_match(&pci_dev->addr);
@@ -740,7 +741,14 @@ mlx5_vdpa_pci_probe(struct rte_pci_driver *pci_drv __rte_unused,
 		priv->num_lag_ports = 1;
 	priv->ctx = ctx;
 	priv->pci_dev = pci_dev;
-	priv->var = mlx5_glue->dv_alloc_var(ctx, 0);
+	for (retry = 0; retry < 7; retry++) {
+		priv->var = mlx5_glue->dv_alloc_var(priv->ctx, 0);
+		if (priv->var != NULL)
+			break;
+		DRV_LOG(WARNING, "Failed to allocate VAR, retry %d.\n", retry);
+		/* Wait Qemu release VAR during vdpa restart, 0.1 sec based. */
+		usleep(100000U << retry);
+	}
 	if (!priv->var) {
 		DRV_LOG(ERR, "Failed to allocate VAR %u.\n", errno);
 		goto error;
-- 
2.33.0

---
  Diff of the applied patch vs upstream commit (please double-check if non-empty:
---
--- -	2021-11-10 14:17:09.880726918 +0800
+++ 0179-vdpa-mlx5-retry-VAR-allocation-during-vDPA-restart.patch	2021-11-10 14:17:01.990745109 +0800
@@ -1 +1 @@
-From 8011a09adda4d99121bcc0fe34c98356559eb32f Mon Sep 17 00:00:00 2001
+From 5497fdbe2b4a9b452b42e7805b0ef7d8395ae86b Mon Sep 17 00:00:00 2001
@@ -4,0 +5,3 @@
+Cc: Xueming Li <xuemingl at nvidia.com>
+
+[ upstream commit 8011a09adda4d99121bcc0fe34c98356559eb32f ]
@@ -16 +18,0 @@
-Cc: stable at dpdk.org
@@ -26 +28 @@
-index 3971f2e335..9a6f64797b 100644
+index 6029cfc3a8..38a93d041d 100644
@@ -29,2 +31 @@
-@@ -506,6 +506,7 @@ mlx5_vdpa_dev_probe(struct mlx5_common_device *cdev)
- {
+@@ -686,6 +686,7 @@ mlx5_vdpa_pci_probe(struct rte_pci_driver *pci_drv __rte_unused,
@@ -32 +33,2 @@
- 	struct mlx5_hca_attr *attr = &cdev->config.hca_attr;
+ 	struct ibv_context *ctx = NULL;
+ 	struct mlx5_hca_attr attr;
@@ -33,0 +36 @@
+ 	int ret;
@@ -35,4 +38,2 @@
- 	if (!attr->vdpa.valid || !attr->vdpa.max_num_virtio_queues) {
- 		DRV_LOG(ERR, "Not enough capabilities to support vdpa, maybe "
-@@ -530,7 +531,14 @@ mlx5_vdpa_dev_probe(struct mlx5_common_device *cdev)
- 	if (attr->num_lag_ports == 0)
+ 	ibv = mlx5_vdpa_get_ib_device_match(&pci_dev->addr);
+@@ -740,7 +741,14 @@ mlx5_vdpa_pci_probe(struct rte_pci_driver *pci_drv __rte_unused,
@@ -40,2 +41,3 @@
- 	priv->cdev = cdev;
--	priv->var = mlx5_glue->dv_alloc_var(priv->cdev->ctx, 0);
+ 	priv->ctx = ctx;
+ 	priv->pci_dev = pci_dev;
+-	priv->var = mlx5_glue->dv_alloc_var(ctx, 0);
@@ -43 +45 @@
-+		priv->var = mlx5_glue->dv_alloc_var(priv->cdev->ctx, 0);
++		priv->var = mlx5_glue->dv_alloc_var(priv->ctx, 0);
@@ -51 +53 @@
- 		DRV_LOG(ERR, "Failed to allocate VAR %u.", errno);
+ 		DRV_LOG(ERR, "Failed to allocate VAR %u.\n", errno);