[PATCH v3 9/9] ci: run reference binaries against current ABI

[David Marchand]

On Mon, 21 Sept 2026 at 18:40, Stephen Hemminger
<stephen at networkplumber.org> wrote:
> Patch 9/9 (ci: run reference binaries against current ABI)
> Errors:
> - The new DPDK_TEST_SKIP computation in .ci/linux-build.sh never
>   skips anything, defeating the mechanism the commit message says
>   it adds ("We may need to skip unit tests that were added since
>   the reference"):
>
>       DPDK_TEST_SKIP=$(grep -vxFf reference/tests.txt build/tests.txt |
>                        sed -n 's,DPDK:.* / ,,p' | tr '\n' ',')
>
>   The sed pattern assumes a "DPDK:<suite> / <test>" format, but
>   `meson test --list --suite fast-tests` never produces that. I
>   built the tree and ran the exact command; with meson 1.12.0 the
>   output is "fast-tests - DPDK:acl_autotest" (dash, no slash). I
>   also checked meson 0.57.2 (DPDK's stated minimum): with a single
>   --suite filter, get_pretty_suite() takes the `else` branch and
>   returns the bare test name only ("acl_autotest", no prefix at
>   all). Neither format contains " / ", so the sed substitution
>   never fires and DPDK_TEST_SKIP is always empty, regardless of
>   meson version.

Let's have AI reply to AI.

  The format change happened between meson 1.9.0 and 1.10.0:
  ┌───────────────┬─────────────────────────────┬──────────────────────┐
  │ Meson Version │        Output Format        │ Sed Pattern Matches?
│
  ├───────────────┼─────────────────────────────┼──────────────────────┤
  │ ≤ 1.9.0       │ DPDK:fast-tests / test_name │ ✅ Yes
│
  ├───────────────┼─────────────────────────────┼──────────────────────┤
  │ ≥ 1.10.0      │ fast-tests - DPDK:test_name │ ❌ No
│
  └───────────────┴─────────────────────────────┴──────────────────────┘
  The sed pattern in .ci/linux-build.sh:255:
  sed -n 's,DPDK:.* / ,,p'

  This pattern looks for DPDK:...something... /  which:
  - Works with meson ≤ 1.9.0 (format: DPDK:fast-tests / acl_autotest)
  - Fails with meson ≥ 1.10.0 (format: fast-tests - DPDK:acl_autotest
— no " / ")

  Where the AI was wrong: The claim about meson 0.57.2 producing bare
names is incorrect. DPDK defines 6 test suites, so len(self.suites) >
1 is always true, and the "pretty" format is always used regardless of
which --suite is queried.

  Current status: Your system has meson 1.7.2, which uses the old
format and works. Ubuntu 22.04 in CI likely has meson ~0.61.x-1.x
which would also work. But the script will break once CI environments
upgrade to meson 1.10.0+.

  Bottom line: The concern is valid for future-proofing, but the
severity is overstated — it doesn't "never work", it works with older
meson but will break with meson ≥ 1.10.0.


>   Consequence: once this lands, any future series that adds a new
>   fast-tests unit test will fail CI when ABI_CHECKS and RUN_TESTS
>   are both enabled -- the new test name is not filtered out, meson
>   test invokes it (DPDK_TEST=<name>) against the old reference
>   dpdk-test binary, which does not implement it, the command exits
>   non-zero into failed="true", and the trailing
>   `[ "$failed" != "true" ]` fails the `-e` script. This is exactly
>   the scenario the DPDK_TEST_SKIP logic was added to prevent.
>
> Info:
> - The commit message says "add a ABI_SKIP_TESTS environment
>   variable in preparation," but the code implements DPDK_TEST_SKIP.
>   Possibly a rebase/rename artifact worth squashing before merge.
>

Ah yes, forgot to remove.


-- 
David Marchand