[PATCH] net/af_xdp: parse numa node id from sysfs

[Du, Frank]

Hi ferruh,

Our application use rte_eth_dev_socket_id to query the socket that a NIC port connected, then allocate lcore/memory according to this affinity.

The remote memory access is really slow compared to local.

Thanks,
Frank

-----Original Message-----
From: Ferruh Yigit, <ferruh.yigit at amd.com> 
Sent: Monday, January 16, 2023 9:15 PM
To: Du, Frank <frank.du at intel.com>; Loftus, Ciara <ciara.loftus at intel.com>
Cc: dev at dpdk.org
Subject: Re: [PATCH] net/af_xdp: parse numa node id from sysfs

On 12/12/2022 12:48 AM, Frank Du wrote:
> Get from /sys/class/net/{if}/device/numa_node.
> 
> Signed-off-by: Frank Du <frank.du at intel.com>
> ---
>  drivers/net/af_xdp/rte_eth_af_xdp.c | 17 ++++++++++++++---
>  1 file changed, 14 insertions(+), 3 deletions(-)
> 
> diff --git a/drivers/net/af_xdp/rte_eth_af_xdp.c 
> b/drivers/net/af_xdp/rte_eth_af_xdp.c
> index b6ec9bf490..38b9d36ab5 100644
> --- a/drivers/net/af_xdp/rte_eth_af_xdp.c
> +++ b/drivers/net/af_xdp/rte_eth_af_xdp.c
> @@ -39,6 +39,7 @@
>  #include <rte_power_intrinsics.h>
>  
>  #include "compat.h"
> +#include "eal_filesystem.h"
>  
>  #ifndef SO_PREFER_BUSY_POLL
>  #define SO_PREFER_BUSY_POLL 69
> @@ -2038,9 +2039,6 @@ rte_pmd_af_xdp_probe(struct rte_vdev_device *dev)
>  		return -EINVAL;
>  	}
>  
> -	if (dev->device.numa_node == SOCKET_ID_ANY)
> -		dev->device.numa_node = rte_socket_id();
> -
>  	if (parse_parameters(kvlist, if_name, &xsk_start_queue_idx,
>  			     &xsk_queue_cnt, &shared_umem, prog_path,
>  			     &busy_budget, &force_copy) < 0) { @@ -2053,6 +2051,19 @@ 
> rte_pmd_af_xdp_probe(struct rte_vdev_device *dev)
>  		return -EINVAL;
>  	}
>  
> +	/* get numa node id from net sysfs */
> +	if (dev->device.numa_node == SOCKET_ID_ANY) {
> +		unsigned long numa = 0;
> +		char numa_path[PATH_MAX];
> +
> +		snprintf(numa_path, sizeof(numa_path), "/sys/class/net/%s/device/numa_node",
> +			 if_name);
> +		if (eal_parse_sysfs_value(numa_path, &numa) != 0)
> +			dev->device.numa_node = rte_socket_id();
> +		else
> +			dev->device.numa_node = numa;
> +	}
> +
>  	busy_budget = busy_budget == -1 ? ETH_AF_XDP_DFLT_BUSY_BUDGET :
>  					busy_budget;
>  

Hi Frank,

It looks reasonable to set virtual DPDK af_xdp device socket to actual underlying device socket. And as I checked quickly, it works as expected.

But what is the impact and motivation of the patch? In other words why you are doing this patch and what output you are expecting as a result?
Did you able to do any performance testing, and are you observing any difference before and after this test?


Thanks,
ferruh