[dpdk-dev] [PATCH v5 0/3] run-time Linking support

[Ananyev, Konstantin]

Hi Xiaouyn,

> -----Original Message-----
> From: Li, Xiaoyun
> Sent: Tuesday, October 3, 2017 4:00 PM
> To: Ananyev, Konstantin <konstantin.ananyev at intel.com>; Richardson, Bruce <bruce.richardson at intel.com>
> Cc: Lu, Wenzhuo <wenzhuo.lu at intel.com>; Zhang, Helin <helin.zhang at intel.com>; dev at dpdk.org; Li, Xiaoyun <xiaoyun.li at intel.com>
> Subject: [PATCH v5 0/3] run-time Linking support
> 
> This patchset dynamically selects functions at run-time based on CPU flags
> that current machine supports.This patchset modifies mempcy, memcpy perf
> test and x86 EFD, using function pointers and bind them at constructor time.
> Then in the cloud environment, users can compiler once for the minimum target
> such as 'haswell'(not 'native') and run on different platforms (equal or above
> haswell) and can get ISA optimization based on running CPU.
> 
> Xiaoyun Li (3):
>   eal/x86: run-time dispatch over memcpy
>   app/test: run-time dispatch over memcpy perf test
>   efd: run-time dispatch over x86 EFD functions
> 
> ---
> v2
> * Use gcc function multi-versioning to avoid compilation issues.
> * Add macros for AVX512 and AVX2. Only if users enable AVX512 and the compiler
> supports it, the AVX512 codes would be compiled. Only if the compiler supports
> AVX2, the AVX2 codes would be compiled.
> 
> v3
> * Reduce function calls via only keep rte_memcpy_xxx.
> * Add conditions that when copy size is small, use inline code path.
> Otherwise, use dynamic code path.
> * To support attribute target, clang version must be greater than 3.7.
> Otherwise, would choose SSE/AVX code path, the same as before.
> * Move two mocro functions to the top of the code since they would be used in
> inline SSE/AVX and dynamic SSE/AVX codes.
> 
> v4
> * Modify rte_memcpy.h to several .c files and modify makefiles to compile
> AVX2 and AVX512 files.
> 
> v5
> * Delete redundant repeated codes of rte_memcpy_xxx.
> * Modify makefiles to enable reuse of existing rte_memcpy.
> * Delete redundant codes of rte_efd_x86.h in v4. Move it into .c file and enable
> compilation -mavx2 for it in makefile since it is already chosen at run-time.
> 

Generally looks good, just two things to fix below.
Konstantin

1. [dpdk-dev,v5,1/3] eal/x86: run-time dispatch over memcpy

Shared target build fails:
http://dpdk.org/ml/archives/test-report/2017-October/031032.html

I think you need to include rte_memcpy_ptr into the:
lib/librte_eal/linuxapp/eal/rte_eal_version.map
lib/librte_eal/bsdapp/eal/rte_eal_version.map
to fix it.

2. [dpdk-dev,v5,3/3] efd: run-time dispatch over x86 EFD functions

/lib/librte_efd/rte_efd_x86.c
....
+efd_value_t
+efd_lookup_internal_avx2(const efd_hashfunc_t *group_hash_idx,
+		const efd_lookuptbl_t *group_lookup_table,
+		const uint32_t hash_val_a, const uint32_t hash_val_b)
+{
+#ifdef CC_SUPPORT_AVX2
+	efd_value_t value = 0;
+	uint32_t i = 0;
+	__m256i vhash_val_a = _mm256_set1_epi32(hash_val_a);
+	__m256i vhash_val_b = _mm256_set1_epi32(hash_val_b);
+
+	for (; i < RTE_EFD_VALUE_NUM_BITS; i += 8) {
+		__m256i vhash_idx =
+				_mm256_cvtepu16_epi32(EFD_LOAD_SI128(
+				(__m128i const *) &group_hash_idx[i]));
+		__m256i vlookup_table = _mm256_cvtepu16_epi32(
+				EFD_LOAD_SI128((__m128i const *)
+				&group_lookup_table[i]));
+		__m256i vhash = _mm256_add_epi32(vhash_val_a,
+				_mm256_mullo_epi32(vhash_idx, vhash_val_b));
+		__m256i vbucket_idx = _mm256_srli_epi32(vhash,
+				EFD_LOOKUPTBL_SHIFT);
+		__m256i vresult = _mm256_srlv_epi32(vlookup_table,
+				vbucket_idx);
+
+		value |= (_mm256_movemask_ps(
+			(__m256) _mm256_slli_epi32(vresult, 31))
+			& ((1 << (RTE_EFD_VALUE_NUM_BITS - i)) - 1)) << i;
+	}
+
+	return value;
+#else

We always build that file with AVX2 option, so I think we can safely  remove
The #ifdef CC_SUPPORT_AVX2 and the code below.

+	RTE_SET_USED(group_hash_idx);
+	RTE_SET_USED(group_lookup_table);
+	RTE_SET_USED(hash_val_a);
+	RTE_SET_USED(hash_val_b);
+	/* Return dummy value, only to avoid compilation breakage */
+	return 0;
+#endif
+
+}


>  lib/librte_eal/bsdapp/eal/Makefile                 |  19 +
>  .../common/include/arch/x86/rte_memcpy.c           |  59 ++
>  .../common/include/arch/x86/rte_memcpy.h           | 861 +------------------
>  .../common/include/arch/x86/rte_memcpy_avx2.c      |  44 +
>  .../common/include/arch/x86/rte_memcpy_avx512f.c   |  44 +
>  .../common/include/arch/x86/rte_memcpy_internal.h  | 909 +++++++++++++++++++++
>  .../common/include/arch/x86/rte_memcpy_sse.c       |  40 +
>  lib/librte_eal/linuxapp/eal/Makefile               |  19 +
>  lib/librte_efd/Makefile                            |   6 +
>  lib/librte_efd/rte_efd_x86.c                       |  87 ++
>  lib/librte_efd/rte_efd_x86.h                       |  48 +-
>  mk/rte.cpuflags.mk                                 |  14 +
>  test/test/test_memcpy_perf.c                       |  40 +-
>  13 files changed, 1285 insertions(+), 905 deletions(-)
>  create mode 100644 lib/librte_eal/common/include/arch/x86/rte_memcpy.c
>  create mode 100644 lib/librte_eal/common/include/arch/x86/rte_memcpy_avx2.c
>  create mode 100644 lib/librte_eal/common/include/arch/x86/rte_memcpy_avx512f.c
>  create mode 100644 lib/librte_eal/common/include/arch/x86/rte_memcpy_internal.h
>  create mode 100644 lib/librte_eal/common/include/arch/x86/rte_memcpy_sse.c
>  create mode 100644 lib/librte_efd/rte_efd_x86.c
> 
> --
> 2.7.4