[dpdk-dev] [PATCH v5 0/3] run-time Linking support

[Li, Xiaoyun]

OK. Will send it later. Many thanks!

> -----Original Message-----
> From: Ananyev, Konstantin
> Sent: Thursday, October 5, 2017 01:56
> To: Li, Xiaoyun <xiaoyun.li at intel.com>; Richardson, Bruce
> <bruce.richardson at intel.com>
> Cc: Lu, Wenzhuo <wenzhuo.lu at intel.com>; Zhang, Helin
> <helin.zhang at intel.com>; dev at dpdk.org
> Subject: RE: [PATCH v5 0/3] run-time Linking support
> 
> Hi Xiaouyn,
> 
> > -----Original Message-----
> > From: Li, Xiaoyun
> > Sent: Tuesday, October 3, 2017 4:00 PM
> > To: Ananyev, Konstantin <konstantin.ananyev at intel.com>; Richardson,
> > Bruce <bruce.richardson at intel.com>
> > Cc: Lu, Wenzhuo <wenzhuo.lu at intel.com>; Zhang, Helin
> > <helin.zhang at intel.com>; dev at dpdk.org; Li, Xiaoyun
> > <xiaoyun.li at intel.com>
> > Subject: [PATCH v5 0/3] run-time Linking support
> >
> > This patchset dynamically selects functions at run-time based on CPU
> > flags that current machine supports.This patchset modifies mempcy,
> > memcpy perf test and x86 EFD, using function pointers and bind them at
> constructor time.
> > Then in the cloud environment, users can compiler once for the minimum
> > target such as 'haswell'(not 'native') and run on different platforms
> > (equal or above
> > haswell) and can get ISA optimization based on running CPU.
> >
> > Xiaoyun Li (3):
> >   eal/x86: run-time dispatch over memcpy
> >   app/test: run-time dispatch over memcpy perf test
> >   efd: run-time dispatch over x86 EFD functions
> >
> > ---
> > v2
> > * Use gcc function multi-versioning to avoid compilation issues.
> > * Add macros for AVX512 and AVX2. Only if users enable AVX512 and the
> > compiler supports it, the AVX512 codes would be compiled. Only if the
> > compiler supports AVX2, the AVX2 codes would be compiled.
> >
> > v3
> > * Reduce function calls via only keep rte_memcpy_xxx.
> > * Add conditions that when copy size is small, use inline code path.
> > Otherwise, use dynamic code path.
> > * To support attribute target, clang version must be greater than 3.7.
> > Otherwise, would choose SSE/AVX code path, the same as before.
> > * Move two mocro functions to the top of the code since they would be
> > used in inline SSE/AVX and dynamic SSE/AVX codes.
> >
> > v4
> > * Modify rte_memcpy.h to several .c files and modify makefiles to
> > compile
> > AVX2 and AVX512 files.
> >
> > v5
> > * Delete redundant repeated codes of rte_memcpy_xxx.
> > * Modify makefiles to enable reuse of existing rte_memcpy.
> > * Delete redundant codes of rte_efd_x86.h in v4. Move it into .c file
> > and enable compilation -mavx2 for it in makefile since it is already chosen
> at run-time.
> >
> 
> Generally looks good, just two things to fix below.
> Konstantin
> 
> 1. [dpdk-dev,v5,1/3] eal/x86: run-time dispatch over memcpy
> 
> Shared target build fails:
> http://dpdk.org/ml/archives/test-report/2017-October/031032.html
> 
> I think you need to include rte_memcpy_ptr into the:
> lib/librte_eal/linuxapp/eal/rte_eal_version.map
> lib/librte_eal/bsdapp/eal/rte_eal_version.map
> to fix it.
> 
> 2. [dpdk-dev,v5,3/3] efd: run-time dispatch over x86 EFD functions
> 
> /lib/librte_efd/rte_efd_x86.c
> ....
> +efd_value_t
> +efd_lookup_internal_avx2(const efd_hashfunc_t *group_hash_idx,
> +		const efd_lookuptbl_t *group_lookup_table,
> +		const uint32_t hash_val_a, const uint32_t hash_val_b)
> { #ifdef
> +CC_SUPPORT_AVX2
> +	efd_value_t value = 0;
> +	uint32_t i = 0;
> +	__m256i vhash_val_a = _mm256_set1_epi32(hash_val_a);
> +	__m256i vhash_val_b = _mm256_set1_epi32(hash_val_b);
> +
> +	for (; i < RTE_EFD_VALUE_NUM_BITS; i += 8) {
> +		__m256i vhash_idx =
> +				_mm256_cvtepu16_epi32(EFD_LOAD_SI128(
> +				(__m128i const *) &group_hash_idx[i]));
> +		__m256i vlookup_table = _mm256_cvtepu16_epi32(
> +				EFD_LOAD_SI128((__m128i const *)
> +				&group_lookup_table[i]));
> +		__m256i vhash = _mm256_add_epi32(vhash_val_a,
> +				_mm256_mullo_epi32(vhash_idx,
> vhash_val_b));
> +		__m256i vbucket_idx = _mm256_srli_epi32(vhash,
> +				EFD_LOOKUPTBL_SHIFT);
> +		__m256i vresult = _mm256_srlv_epi32(vlookup_table,
> +				vbucket_idx);
> +
> +		value |= (_mm256_movemask_ps(
> +			(__m256) _mm256_slli_epi32(vresult, 31))
> +			& ((1 << (RTE_EFD_VALUE_NUM_BITS - i)) - 1)) << i;
> +	}
> +
> +	return value;
> +#else
> 
> We always build that file with AVX2 option, so I think we can safely  remove
> The #ifdef CC_SUPPORT_AVX2 and the code below.
> 
> +	RTE_SET_USED(group_hash_idx);
> +	RTE_SET_USED(group_lookup_table);
> +	RTE_SET_USED(hash_val_a);
> +	RTE_SET_USED(hash_val_b);
> +	/* Return dummy value, only to avoid compilation breakage */
> +	return 0;
> +#endif
> +
> +}
> 
> 
> >  lib/librte_eal/bsdapp/eal/Makefile                 |  19 +
> >  .../common/include/arch/x86/rte_memcpy.c           |  59 ++
> >  .../common/include/arch/x86/rte_memcpy.h           | 861 +------------------
> >  .../common/include/arch/x86/rte_memcpy_avx2.c      |  44 +
> >  .../common/include/arch/x86/rte_memcpy_avx512f.c   |  44 +
> >  .../common/include/arch/x86/rte_memcpy_internal.h  | 909
> +++++++++++++++++++++
> >  .../common/include/arch/x86/rte_memcpy_sse.c       |  40 +
> >  lib/librte_eal/linuxapp/eal/Makefile               |  19 +
> >  lib/librte_efd/Makefile                            |   6 +
> >  lib/librte_efd/rte_efd_x86.c                       |  87 ++
> >  lib/librte_efd/rte_efd_x86.h                       |  48 +-
> >  mk/rte.cpuflags.mk                                 |  14 +
> >  test/test/test_memcpy_perf.c                       |  40 +-
> >  13 files changed, 1285 insertions(+), 905 deletions(-)  create mode
> > 100644 lib/librte_eal/common/include/arch/x86/rte_memcpy.c
> >  create mode 100644
> > lib/librte_eal/common/include/arch/x86/rte_memcpy_avx2.c
> >  create mode 100644
> > lib/librte_eal/common/include/arch/x86/rte_memcpy_avx512f.c
> >  create mode 100644
> > lib/librte_eal/common/include/arch/x86/rte_memcpy_internal.h
> >  create mode 100644
> > lib/librte_eal/common/include/arch/x86/rte_memcpy_sse.c
> >  create mode 100644 lib/librte_efd/rte_efd_x86.c
> >
> > --
> > 2.7.4