[PATCH v5 04/10] app/test: build using per-file dependency matrix

[Bruce Richardson]

On Tue, Aug 15, 2023 at 03:05:08PM -0400, Patrick Robb wrote:
>    Adam from our team just raised something important about this patch and
>    UNH CI which I missed during the RFC discussion.
>    Presently, eal_flags_file_prefix_autotest fails on arm tx2 systems (arm
>    people are aware of this, they think it might be a memory leak from
>    mlx5 driver), so at their request we are disabling this unit test (on
>    arm only) for them for the time being. This failure was discovered when
>    we were initially standing up arm unit testing on tx2 servers earlier
>    this year - previously there was no coverage for this. We do this
>    filtering by going through the /app/test/meson.build file and resetting
>    the eal_flags_file_prefix_autotest line to an empty string. This
>    process is broken by your patch series. Again, I'm sorry I didn't catch
>    this concern when discussing it during the RFC. This is why you are not
>    getting unit test results for arm64, we can't run unit tests given the
>    changes in this patch.

If this is a (semi)permanent issue, would it not be better to actually
patch in the disabling of this test into the test sources, i.e. if ARM64
have the test return TEST_SKIPPED?

>    Your refactor likely means that going forward, we will no longer be
>    able to tailor the fast test suite (or any suite) per vendor request.
>    That might actually be a good thing. In any case, if this is merged
>    now, it is going to put the tree in a state where our CI doesn't run
>    any unit testing on ARM. I don't know how close this patch series is to
>    possibly hitting mainline, but, if possible, can that be delayed,
>    pending us figuring out how we will respond to this situation?

There is at least one more issue I am aware of with this set - the failing
mempool tests - which needs to be fixed, so there will be at least one more
version.

If this is likely a common occurrance where we need to disable one
particular test for a particular platform, would it be worth designing such
a feature into the rework. For example, it should be fairly easy to have
the testname-extraction script read a blocklist from the environment, and
omit informing meson of those test cases. Would that work?

/Bruce