[PATCH 16/18] test: allow setting random seed for tests

[Stephen Hemminger]

Many tests build their input data with rte_rand(), which is seeded
from the system entropy source and therefore produces a different
sequence on every run. Varying the data is good for coverage, but it
adds noise when comparing performance results between two builds.

Add a DPDK_TEST_SEED environment variable to the test application that
seeds the generator with a fixed value. The generator is reseeded
before each test is dispatched, so a test gets the same input data no
matter what ran before it in the same invocation. When the variable is
not set the behaviour is unchanged and each run gets a random seed.

Note that rand_perf_autotest calls rte_srand() itself. It measures
generator latency rather than any value derived from the sequence, so
it is left alone.

Signed-off-by: Stephen Hemminger <stephen at networkplumber.org>
---
 app/test/test.c                        | 48 ++++++++++++++++++++++++++
 doc/guides/contributing/unit_test.rst  | 15 ++++++++
 doc/guides/rel_notes/release_26_11.rst |  2 ++
 3 files changed, 65 insertions(+)

diff --git a/app/test/test.c b/app/test/test.c
index c610c3588e..176e4fa1f5 100644
--- a/app/test/test.c
+++ b/app/test/test.c
@@ -6,6 +6,7 @@
 #include <stdio.h>
 #include <stdint.h>
 #include <stdarg.h>
+#include <stdbool.h>
 #include <stdlib.h>
 #include <errno.h>
 #include <ctype.h>
@@ -21,6 +22,7 @@ extern cmdline_parse_ctx_t main_ctx[];
 #include <rte_eal.h>
 #include <rte_cycles.h>
 #include <rte_log.h>
+#include <rte_random.h>
 #include <rte_string_fns.h>
 #ifdef RTE_LIB_TIMER
 #include <rte_timer.h>
@@ -107,6 +109,33 @@ do_recursive_call(void)
 
 int last_test_result;
 
+/* Tests that use rte_rand() get a different sequence on every run, which
+ * is good for coverage but makes it hard to compare performance results.
+ * Setting DPDK_TEST_SEED gives a repeatable sequence instead.
+ */
+static bool use_test_seed;
+static uint64_t test_seed;
+
+static int
+get_test_seed(void)
+{
+	const char *env = getenv("DPDK_TEST_SEED");
+	char *end;
+
+	if (env == NULL || *env == '\0')
+		return 0;
+
+	errno = 0;
+	test_seed = strtoull(env, &end, 0);
+	if (errno != 0 || end == env || *end != '\0') {
+		fprintf(stderr, "Invalid DPDK_TEST_SEED: '%s'\n", env);
+		return -1;
+	}
+
+	use_test_seed = true;
+	return 0;
+}
+
 #define MAX_EXTRA_ARGS 32
 
 int
@@ -175,8 +204,18 @@ main(int argc, char **argv)
 		goto out;
 	}
 
+	if (get_test_seed() < 0) {
+		ret = -1;
+		goto out;
+	}
+
 	recursive_call = getenv(RECURSIVE_ENV_VAR);
 	if (recursive_call != NULL) {
+		/* Child instances inherit the environment, so seed them
+		 * too rather than leaving them randomly seeded.
+		 */
+		if (use_test_seed)
+			rte_srand(test_seed);
 		ret = do_recursive_call();
 		goto out;
 	}
@@ -233,6 +272,12 @@ main(int argc, char **argv)
 				}
 			}
 
+			/* Reseed before each test so that a test gets the same
+			 * sequence no matter what ran before it.
+			 */
+			if (use_test_seed)
+				rte_srand(test_seed);
+
 			snprintf(buf, sizeof(buf), "%s\n", tests[i]);
 			if (cmdline_parse_check(cl, buf) < 0) {
 				printf("Error: invalid test command: '%s'\n", tests[i]);
@@ -260,6 +305,9 @@ main(int argc, char **argv)
 			goto out;
 		}
 
+		if (use_test_seed)
+			rte_srand(test_seed);
+
 		cmdline_interact(cl);
 		cmdline_stdin_exit(cl);
 	}
diff --git a/doc/guides/contributing/unit_test.rst b/doc/guides/contributing/unit_test.rst
index f13ddff291..5d08fe1792 100644
--- a/doc/guides/contributing/unit_test.rst
+++ b/doc/guides/contributing/unit_test.rst
@@ -114,6 +114,21 @@ via the ``DPDK_TEST_PARAMS`` argument,
 in case some tests need additional configuration.
 This is not currently used in the Meson test suites.
 
+Many tests build their input data with ``rte_rand()``,
+which produces a different sequence on every run.
+This is useful for coverage, but it adds noise when comparing results.
+Setting ``DPDK_TEST_SEED`` to an integer seeds the generator with that
+value before each test command is run, so that a test gets the same input
+data no matter what ran before it::
+
+   $ DPDK_TEST_SEED=42 ./build/app/test/dpdk-test lpm_perf_autotest
+
+This also works when running the tests via Meson::
+
+   $ DPDK_TEST_SEED=42 meson test -C build --suite perf-tests
+
+When the variable is not set, the generator keeps its default random seeding.
+
 
 Running test cases via Meson
 ----------------------------
diff --git a/doc/guides/rel_notes/release_26_11.rst b/doc/guides/rel_notes/release_26_11.rst
index 9d1238d6f3..d315d426f4 100644
--- a/doc/guides/rel_notes/release_26_11.rst
+++ b/doc/guides/rel_notes/release_26_11.rst
@@ -69,6 +69,8 @@ New Features
     64 bits of the value passed to ``rte_srand()`` now affect the state,
     so a given seed produces a different sequence than in previous
     releases.
+  * The test application now accepts a ``DPDK_TEST_SEED`` environment
+    variable to make runs using ``rte_rand()`` repeatable.
 
 
 Removed Items
-- 
2.53.0