[dpdk-dev] [PATCH v3 7/7] test/hash: read-write lock-free concurrency test

Wang, Yipeng1 yipeng1.wang at intel.com
Sat Oct 13 04:52:52 CEST 2018


>-----Original Message-----
>From: Honnappa Nagarahalli [mailto:honnappa.nagarahalli at arm.com]
>Sent: Thursday, October 11, 2018 11:32 PM
>To: Richardson, Bruce <bruce.richardson at intel.com>; De Lara Guarch, Pablo <pablo.de.lara.guarch at intel.com>
>Cc: dev at dpdk.org; Wang, Yipeng1 <yipeng1.wang at intel.com>; honnappa.nagarahalli at arm.com; dharmik.thakkar at arm.com;
>gavin.hu at arm.com; nd at arm.com
>Subject: [PATCH v3 7/7] test/hash: read-write lock-free concurrency test
>
>From: Dharmik Thakkar <dharmik.thakkar at arm.com>
>
>Unit tests to check for hash lookup perf with lock-free enabled and
>with lock-free disabled.
>Unit tests performed with readers running in parallel with writers.
>
>Tests include:
>
>- hash lookup on existing keys with:
>  - hash add causing NO key-shifts of existing keys in the table
>
>- hash lookup on existing keys likely to be on shift-path with:
>  - hash add causing key-shifts of existing keys in the table
>
>- hash lookup on existing keys NOT likely to be on shift-path with:
>  - hash add causing key-shifts of existing keys in the table
>
>- hash lookup on non-existing keys with:
>  - hash add causing NO key-shifts of existing keys in the table
>  - hash add causing key-shifts of existing keys in the table
>
>- hash lookup on keys likely to be on shift-path with:
>  - multiple writers causing key-shifts of existing keys in the table
>
>Signed-off-by: Dharmik Thakkar <dharmik.thakkar at arm.com>
>Reviewed-by: Honnappa Nagarahalli <honnappa.nagarahalli at arm.com>
>Reviewed-by: Gavin Hu <gavin.hu at arm.com>
>---
> test/test/Makefile                 |    1 +
> test/test/meson.build              |    1 +
> test/test/test_hash_readwrite_lf.c | 1084 ++++++++++++++++++++++++++++++++++++
> 3 files changed, 1086 insertions(+)
> create mode 100644 test/test/test_hash_readwrite_lf.c
>
>diff --git a/test/test/Makefile b/test/test/Makefile
>index 5d8b1dc..0e0e6c4 100644
>--- a/test/test/Makefile
>+++ b/test/test/Makefile
>@@ -116,6 +116,7 @@ SRCS-$(CONFIG_RTE_LIBRTE_HASH) += test_hash_functions.c
> SRCS-$(CONFIG_RTE_LIBRTE_HASH) += test_hash_scaling.c
> SRCS-$(CONFIG_RTE_LIBRTE_HASH) += test_hash_multiwriter.c
> SRCS-$(CONFIG_RTE_LIBRTE_HASH) += test_hash_readwrite.c
>+SRCS-$(CONFIG_RTE_LIBRTE_HASH) += test_hash_readwrite_lf.c
>
> SRCS-$(CONFIG_RTE_LIBRTE_LPM) += test_lpm.c
> SRCS-$(CONFIG_RTE_LIBRTE_LPM) += test_lpm_perf.c
>diff --git a/test/test/meson.build b/test/test/meson.build
>index d696f5e..bc3350f 100644
>--- a/test/test/meson.build
>+++ b/test/test/meson.build
>@@ -46,6 +46,7 @@ test_sources = files('commands.c',
> 	'test_hash_multiwriter.c',
> 	'test_hash_readwrite.c',
> 	'test_hash_perf.c',
>+	'test_hash_readwrite_lf.c',
> 	'test_hash_scaling.c',
> 	'test_interrupts.c',
> 	'test_kni.c',
>diff --git a/test/test/test_hash_readwrite_lf.c b/test/test/test_hash_readwrite_lf.c
>new file mode 100644
>index 0000000..841e989
>--- /dev/null
>+++ b/test/test/test_hash_readwrite_lf.c
>@@ -0,0 +1,1084 @@
>+/* SPDX-License-Identifier: BSD-3-Clause
>+ * Copyright(c) 2018 Arm Limited
>+ */
>+
>+#include <inttypes.h>
>+#include <locale.h>
>+
>+#include <rte_cycles.h>
>+#include <rte_hash.h>
>+#include <rte_hash_crc.h>
>+#include <rte_jhash.h>
>+#include <rte_launch.h>
>+#include <rte_malloc.h>
>+#include <rte_random.h>
>+#include <rte_spinlock.h>
>+
>+#include "test.h"
>+
>+#ifndef RTE_HASH_EXTRA_FLAGS_RW_CONCURRENCY_LF
>+#define RTE_HASH_EXTRA_FLAGS_RW_CONCURRENCY_LF 0
>+#endif
>+
>+#define RUN_WITH_HTM_DISABLED 0
>+
>+#if (RUN_WITH_HTM_DISABLED)
>+
>+#define TOTAL_ENTRY (5*1024)
>+#define TOTAL_INSERT (5*1024)
>+
>+#else
>+
>+#define TOTAL_ENTRY (16*1024*1024)
>+#define TOTAL_INSERT (16*1024*1024)
[Wang, Yipeng] 
You can make the number smaller since a previous patch on multi-write test
suggests to use a smaller number (how about 5 million) to reduce the total testing time.
>+
>+static rte_atomic64_t gread_cycles;
>+static rte_atomic64_t greads;
>+
>+static volatile uint8_t writer_done;
>+static volatile uint8_t multi_writer_done[4];
>+uint8_t num_test;
>+uint8_t htm;
[Wang, Yipeng] It would be better to make the variables local.
>+
>+	for (i = 0; i < count_keys_no_ks; i++) {
>+		/* Identify keys in keys_no_ks with value less than 1M */
[Wang, Yipeng] TOTAL_INSERT is 16M. Other comments too.
>+		if (keys_no_ks[i] < TOTAL_INSERT)
>+			found[keys_no_ks[i]]++;
>+	}
>+
>+	for (i = 0; i < count_keys_ks; i++) {
>+		/* Identify keys in keys_ks with value less than 1M */
>+		if (keys_ks[i] < TOTAL_INSERT)
>+			found[keys_ks[i]]++;
>+	}
>+
>+	uint32_t count_keys_absent = 0;
>+	for (i = 0; i < TOTAL_INSERT; i++) {
>+		/* Identify missing keys between 0 and 1M */
>+/*
>+ * Test lookup perf:
[Wang, Yipeng] add comment: for multi-writer
>+ * Reader(s) lookup keys present in the table and likely on the shift-path while
>+ * Writers add keys causing key-shifts.
>+ */
>+static int
>+test_hash_multi_add_lookup(struct rwc_perf *rwc_perf_results, int rwc_lf)
>+{
>+	unsigned int n, m;
>+	uint64_t i;
>+static int
>+test_hash_readwrite_lf_main(void)
>+{
>+	/*
>+	 * Variables used to choose different tests.
>+	 * rwc_lf indicates if read-write concurrency lock-free support is
>+	 * enabled.
>+	 * htm indicates if Hardware transactional memory support is enabled.
>+	 */
>+	int rwc_lf = 0;
>+	int use_jhash = 0;
>+	num_test = NUM_TEST;
>+	if (rte_lcore_count() == 1) {
>+		printf("More than one lcore is required "
>+			"to do read write lock-free concurrency test\n");
>+		return -1;
>+	}
>+
>+	setlocale(LC_NUMERIC, "");
>+
>+	if (rte_tm_supported())
>+		htm = 1;
>+	else
>+		htm = 0;
>+
>+	if (init_params(rwc_lf, use_jhash) != 0)
>+		return -1;
>+	if (generate_keys() != 0)
[Wang, Yipeng] Generate_keys may run for a long time, so print something to indicate
the test has started.
>+		return -1;
>+	if (get_enabled_cores_list() != 0)
>+		return -1;
>+
[Wang, Yipeng]
Please change/add multi-lookup(bulk lookup) to all the tests. Bulk lookup is usually more commonly used and people care about
as a performance test.
It would make sense to change other multi-thread performance unit test to use bulk lookup as well in future. 




More information about the dev mailing list