[dpdk-dev] [PATCH v2] ethdev: fix representor port ID search by name
Xueming(Steven) Li
xuemingl at nvidia.com
Fri Aug 27 11:18:03 CEST 2021
> -----Original Message-----
> From: Andrew Rybchenko <andrew.rybchenko at oktetlabs.ru>
> Sent: Wednesday, August 18, 2021 10:00 PM
> To: Ajit Khaparde <ajit.khaparde at broadcom.com>; Somnath Kotur <somnath.kotur at broadcom.com>; John Daley
> <johndale at cisco.com>; Hyong Youb Kim <hyonkim at cisco.com>; Beilei Xing <beilei.xing at intel.com>; Qiming Yang
> <qiming.yang at intel.com>; Qi Zhang <qi.z.zhang at intel.com>; Haiyue Wang <haiyue.wang at intel.com>; Matan Azrad
> <matan at nvidia.com>; Shahaf Shuler <shahafs at nvidia.com>; Slava Ovsiienko <viacheslavo at nvidia.com>; NBU-Contact-Thomas
> Monjalon <thomas at monjalon.net>; Ferruh Yigit <ferruh.yigit at intel.com>
> Cc: dev at dpdk.org; Viacheslav Galaktionov <viacheslav.galaktionov at oktetlabs.ru>; Xueming(Steven) Li <xuemingl at nvidia.com>
> Subject: [PATCH v2] ethdev: fix representor port ID search by name
>
> From: Viacheslav Galaktionov <viacheslav.galaktionov at oktetlabs.ru>
>
> Getting a list of representors from a representor does not make sense.
> Instead, a parent device should be used.
>
> To this end, extend the rte_eth_dev_data structure to include the port ID of the parent device for representors.
>
> Signed-off-by: Viacheslav Galaktionov <viacheslav.galaktionov at oktetlabs.ru>
> Signed-off-by: Andrew Rybchenko <andrew.rybchenko at oktetlabs.ru>
> ---
> The new field is added into the hole in rte_eth_dev_data structure.
> The patch does not change ABI, but extra care is required since ABI check is disabled for the structure because of the libabigail bug [1].
>
> Potentially it is bad for out-of-tree drivers which implement representors but do not fill in a new parert_port_id field in
> rte_eth_dev_data structure. Do we care?
>
> May be the patch should add lines to release notes, but I'd like to get initial feedback first.
>
> mlx5 changes should be reviwed by maintainers very carefully, since we are not sure if we patch it correctly.
>
> [1] https://sourceware.org/bugzilla/show_bug.cgi?id=28060
>
> drivers/net/bnxt/bnxt_reps.c | 1 +
> drivers/net/enic/enic_vf_representor.c | 1 +
> drivers/net/i40e/i40e_vf_representor.c | 1 +
> drivers/net/ice/ice_dcf_vf_representor.c | 1 + drivers/net/ixgbe/ixgbe_vf_representor.c | 1 +
> drivers/net/mlx5/linux/mlx5_os.c | 17 +++++++++++++++++
> drivers/net/mlx5/windows/mlx5_os.c | 17 +++++++++++++++++
> lib/ethdev/ethdev_driver.h | 6 +++---
> lib/ethdev/rte_class_eth.c | 22 ++++++++++++++++++++--
> lib/ethdev/rte_ethdev.c | 8 ++++----
> lib/ethdev/rte_ethdev_core.h | 4 ++++
> 11 files changed, 70 insertions(+), 9 deletions(-)
>
> diff --git a/drivers/net/bnxt/bnxt_reps.c b/drivers/net/bnxt/bnxt_reps.c index bdbad53b7d..902591cd39 100644
> --- a/drivers/net/bnxt/bnxt_reps.c
> +++ b/drivers/net/bnxt/bnxt_reps.c
> @@ -187,6 +187,7 @@ int bnxt_representor_init(struct rte_eth_dev *eth_dev, void *params)
> eth_dev->data->dev_flags |= RTE_ETH_DEV_REPRESENTOR |
> RTE_ETH_DEV_AUTOFILL_QUEUE_XSTATS;
> eth_dev->data->representor_id = rep_params->vf_id;
> + eth_dev->data->parent_port_id = rep_params->parent_dev->data->port_id;
>
> rte_eth_random_addr(vf_rep_bp->dflt_mac_addr);
> memcpy(vf_rep_bp->mac_addr, vf_rep_bp->dflt_mac_addr, diff --git a/drivers/net/enic/enic_vf_representor.c
> b/drivers/net/enic/enic_vf_representor.c
> index 79dd6e5640..6ee7967ce9 100644
> --- a/drivers/net/enic/enic_vf_representor.c
> +++ b/drivers/net/enic/enic_vf_representor.c
> @@ -662,6 +662,7 @@ int enic_vf_representor_init(struct rte_eth_dev *eth_dev, void *init_params)
> eth_dev->data->dev_flags |= RTE_ETH_DEV_REPRESENTOR |
> RTE_ETH_DEV_AUTOFILL_QUEUE_XSTATS;
> eth_dev->data->representor_id = vf->vf_id;
> + eth_dev->data->parent_port_id = pf->port_id;
> eth_dev->data->mac_addrs = rte_zmalloc("enic_mac_addr_vf",
> sizeof(struct rte_ether_addr) *
> ENIC_UNICAST_PERFECT_FILTERS, 0);
> diff --git a/drivers/net/i40e/i40e_vf_representor.c b/drivers/net/i40e/i40e_vf_representor.c
> index 0481b55381..865b637585 100644
> --- a/drivers/net/i40e/i40e_vf_representor.c
> +++ b/drivers/net/i40e/i40e_vf_representor.c
> @@ -514,6 +514,7 @@ i40e_vf_representor_init(struct rte_eth_dev *ethdev, void *init_params)
> ethdev->data->dev_flags |= RTE_ETH_DEV_REPRESENTOR |
> RTE_ETH_DEV_AUTOFILL_QUEUE_XSTATS;
> ethdev->data->representor_id = representor->vf_id;
> + ethdev->data->parent_port_id = pf->dev_data->parent_port_id;
>
> /* Setting the number queues allocated to the VF */
> ethdev->data->nb_rx_queues = vf->vsi->nb_qps; diff --git a/drivers/net/ice/ice_dcf_vf_representor.c
> b/drivers/net/ice/ice_dcf_vf_representor.c
> index 970461f3e9..c7cd3fd290 100644
> --- a/drivers/net/ice/ice_dcf_vf_representor.c
> +++ b/drivers/net/ice/ice_dcf_vf_representor.c
> @@ -418,6 +418,7 @@ ice_dcf_vf_repr_init(struct rte_eth_dev *vf_rep_eth_dev, void *init_param)
>
> vf_rep_eth_dev->data->dev_flags |= RTE_ETH_DEV_REPRESENTOR;
> vf_rep_eth_dev->data->representor_id = repr->vf_id;
> + vf_rep_eth_dev->data->parent_port_id =
> +repr->dcf_eth_dev->data->port_id;
>
> vf_rep_eth_dev->data->mac_addrs = &repr->mac_addr;
>
> diff --git a/drivers/net/ixgbe/ixgbe_vf_representor.c b/drivers/net/ixgbe/ixgbe_vf_representor.c
> index d5b636a194..7a2063849e 100644
> --- a/drivers/net/ixgbe/ixgbe_vf_representor.c
> +++ b/drivers/net/ixgbe/ixgbe_vf_representor.c
> @@ -197,6 +197,7 @@ ixgbe_vf_representor_init(struct rte_eth_dev *ethdev, void *init_params)
>
> ethdev->data->dev_flags |= RTE_ETH_DEV_REPRESENTOR;
> ethdev->data->representor_id = representor->vf_id;
> + ethdev->data->parent_port_id = representor->pf_ethdev->data->port_id;
>
> /* Set representor device ops */
> ethdev->dev_ops = &ixgbe_vf_representor_dev_ops; diff --git a/drivers/net/mlx5/linux/mlx5_os.c
> b/drivers/net/mlx5/linux/mlx5_os.c
> index 5f8766aa48..a68fa7beb7 100644
> --- a/drivers/net/mlx5/linux/mlx5_os.c
> +++ b/drivers/net/mlx5/linux/mlx5_os.c
> @@ -1677,6 +1677,23 @@ mlx5_dev_spawn(struct rte_device *dpdk_dev,
> if (priv->representor) {
> eth_dev->data->dev_flags |= RTE_ETH_DEV_REPRESENTOR;
> eth_dev->data->representor_id = priv->representor_id;
> + MLX5_ETH_FOREACH_DEV(port_id, priv->pci_dev) {
> + struct mlx5_priv *opriv =
> + rte_eth_devices[port_id].data->dev_private;
> + if (opriv &&
> + opriv->master &&
> + opriv->domain_id == priv->domain_id &&
> + opriv->sh == priv->sh) {
> + eth_dev->data->parent_port_id =
> + rte_eth_devices[port_id].data->port_id;
> + break;
> + }
> + }
> + if (port_id >= RTE_MAX_ETHPORTS) {
> + DRV_LOG(ERR, "no master device for representor");
> + err = ENODEV;
> + goto error;
> + }
> }
> priv->mp_id.port_id = eth_dev->data->port_id;
> strlcpy(priv->mp_id.name, MLX5_MP_NAME, RTE_MP_MAX_NAME_LEN); diff --git a/drivers/net/mlx5/windows/mlx5_os.c
> b/drivers/net/mlx5/windows/mlx5_os.c
> index 7e1df1c751..0c5a02bfcb 100644
> --- a/drivers/net/mlx5/windows/mlx5_os.c
> +++ b/drivers/net/mlx5/windows/mlx5_os.c
> @@ -543,6 +543,23 @@ mlx5_dev_spawn(struct rte_device *dpdk_dev,
> if (priv->representor) {
> eth_dev->data->dev_flags |= RTE_ETH_DEV_REPRESENTOR;
> eth_dev->data->representor_id = priv->representor_id;
> + MLX5_ETH_FOREACH_DEV(port_id, priv->pci_dev) {
> + struct mlx5_priv *opriv =
> + rte_eth_devices[port_id].data->dev_private;
> + if (opriv &&
> + opriv->master &&
> + opriv->domain_id == priv->domain_id &&
> + opriv->sh == priv->sh) {
> + eth_dev->data->parent_port_id =
> + rte_eth_devices[port_id].data->port_id;
Could this value different than port_id?
> + break;
> + }
> + }
> + if (port_id >= RTE_MAX_ETHPORTS) {
> + DRV_LOG(ERR, "no master device for representor");
> + err = ENODEV;
> + goto error;
Here shouldn't be an error.
Parent port ID default to 0, it could be wrong if multiple PF probed, let's default to current port ID.
> + }
> }
> /*
> * Store associated network device interface index. This index diff --git a/lib/ethdev/ethdev_driver.h
> b/lib/ethdev/ethdev_driver.h index fd5b7ca550..d1a1499538 100644
> --- a/lib/ethdev/ethdev_driver.h
> +++ b/lib/ethdev/ethdev_driver.h
> @@ -1287,8 +1287,8 @@ struct rte_eth_devargs {
> * For backward compatibility, if no representor info, direct
> * map legacy VF (no controller and pf).
> *
> - * @param ethdev
> - * Handle of ethdev port.
> + * @param port_id
> + * Port ID of the backing device.
> * @param type
> * Representor type.
> * @param controller
> @@ -1305,7 +1305,7 @@ struct rte_eth_devargs {
> */
> __rte_internal
> int
> -rte_eth_representor_id_get(const struct rte_eth_dev *ethdev,
> +rte_eth_representor_id_get(uint16_t port_id,
> enum rte_eth_representor_type type,
> int controller, int pf, int representor_port,
> uint16_t *repr_id);
> diff --git a/lib/ethdev/rte_class_eth.c b/lib/ethdev/rte_class_eth.c index 1fe5fa1f36..167d2d798c 100644
> --- a/lib/ethdev/rte_class_eth.c
> +++ b/lib/ethdev/rte_class_eth.c
> @@ -95,14 +95,32 @@ eth_representor_cmp(const char *key __rte_unused,
> c = i / (np * nf);
> p = (i / nf) % np;
> f = i % nf;
> - if (rte_eth_representor_id_get(edev,
> + /*
> + * rte_eth_representor_id_get expects to receive port ID of
> + * the master device, but in order to maintain compatibility
> + * with mlx5's hardware bonding and legacy representor
> + * specification using just VF numbers, the representor's port
> + * ID is tried first.
> + */
> + ret = rte_eth_representor_id_get(edev->data->port_id,
> eth_da.type,
> eth_da.nb_mh_controllers == 0 ? -1 :
> eth_da.mh_controllers[c],
> eth_da.nb_ports == 0 ? -1 : eth_da.ports[p],
> eth_da.nb_representor_ports == 0 ? -1 :
> eth_da.representor_ports[f],
> - &id) < 0)
> + &id);
> + if (ret == -ENOTSUP)
> + ret = rte_eth_representor_id_get(
> + edev->data->parent_port_id,
> + eth_da.type,
> + eth_da.nb_mh_controllers == 0 ? -1 :
> + eth_da.mh_controllers[c],
> + eth_da.nb_ports == 0 ? -1 : eth_da.ports[p],
> + eth_da.nb_representor_ports == 0 ? -1 :
> + eth_da.representor_ports[f],
> + &id);
> + if (ret < 0)
> continue;
> if (data->representor_id == id)
> return 0;
> diff --git a/lib/ethdev/rte_ethdev.c b/lib/ethdev/rte_ethdev.c index 9d95cd11e1..228ef7bf23 100644
> --- a/lib/ethdev/rte_ethdev.c
> +++ b/lib/ethdev/rte_ethdev.c
> @@ -5997,7 +5997,7 @@ rte_eth_devargs_parse(const char *dargs, struct rte_eth_devargs *eth_da) }
>
> int
> -rte_eth_representor_id_get(const struct rte_eth_dev *ethdev,
> +rte_eth_representor_id_get(uint16_t port_id,
> enum rte_eth_representor_type type,
> int controller, int pf, int representor_port,
> uint16_t *repr_id)
> @@ -6013,7 +6013,7 @@ rte_eth_representor_id_get(const struct rte_eth_dev *ethdev,
> return -EINVAL;
>
> /* Get PMD representor range info. */
> - ret = rte_eth_representor_info_get(ethdev->data->port_id, NULL);
> + ret = rte_eth_representor_info_get(port_id, NULL);
> if (ret == -ENOTSUP && type == RTE_ETH_REPRESENTOR_VF &&
> controller == -1 && pf == -1) {
> /* Direct mapping for legacy VF representor. */ @@ -6028,7 +6028,7 @@ rte_eth_representor_id_get(const struct
> rte_eth_dev *ethdev,
> if (info == NULL)
> return -ENOMEM;
> info->nb_ranges_alloc = n;
> - ret = rte_eth_representor_info_get(ethdev->data->port_id, info);
> + ret = rte_eth_representor_info_get(port_id, info);
> if (ret < 0)
> goto out;
>
> @@ -6047,7 +6047,7 @@ rte_eth_representor_id_get(const struct rte_eth_dev *ethdev,
> continue;
> if (info->ranges[i].id_end < info->ranges[i].id_base) {
> RTE_LOG(WARNING, EAL, "Port %hu invalid representor ID Range %u - %u, entry %d\n",
> - ethdev->data->port_id, info->ranges[i].id_base,
> + port_id, info->ranges[i].id_base,
> info->ranges[i].id_end, i);
> continue;
>
> diff --git a/lib/ethdev/rte_ethdev_core.h b/lib/ethdev/rte_ethdev_core.h index edf96de2dc..13cb84b52f 100644
> --- a/lib/ethdev/rte_ethdev_core.h
> +++ b/lib/ethdev/rte_ethdev_core.h
> @@ -185,6 +185,10 @@ struct rte_eth_dev_data {
> /**< Switch-specific identifier.
> * Valid if RTE_ETH_DEV_REPRESENTOR in dev_flags.
> */
> + uint16_t parent_port_id;
> + /**< Port ID of the backing device.
> + * Valid if RTE_ETH_DEV_REPRESENTOR in dev_flags.
> + */
>
> pthread_mutex_t flow_ops_mutex; /**< rte_flow ops mutex. */
> uint64_t reserved_64s[4]; /**< Reserved for future fields */
> --
> 2.30.2
More information about the dev
mailing list