[PATCH v5 04/10] app/test: build using per-file dependency matrix

[David Marchand]

On Wed, Aug 16, 2023 at 8:30 PM Patrick Robb <probb at iol.unh.edu> wrote:
>> Patching sources from the test tool is a poor solution.
>> In general, developers won't be aware of source patching and will
>> waste time trying to understand why they can't reproduce what the CI
>> reports (it happened to me with DTS on the interrupt stuff with vhost,
>> at least).
>>
>> For this specific case of skipping a test, if nobody can fix the
>> issue, I prefer if the CI can skip some "known broken in my lab" tests
>> via some meson configuration.
>> And, such configuration should be easy to catch in the test report.
>>
> I strongly agree on all points, which is why I said it was probably a good thing anyhow for us to lose this ability. In the case of the disabled fast-test for arm, that was a new discovery coming from adding new environments, not a regression introduced by a patch, and I don't think it made sense then to block the introduction of the entire unit test coverage for arm while they looked into this issue. If it's possible to introduce meson configure functionality to disable specific tests, that does give us more flexibility. And it's obviously a better process than us doing it at the CI end.
>
> We don't currently patch source in any other way in our CI testing.

It is possible to list tests from meson.
Why not simply list the existing tests and filter the non working one?

$ meson test -C <build_dir> --suite fast-tests --list
$ meson test -C <build_dir> <explicit list> --test-args ...

That requires no change in DPDK.


-- 
David Marchand