[dpdk-dev] [PATCH] test: allow taking test names from commandline

[David Marchand]

On Tue, Apr 13, 2021 at 6:49 PM Bruce Richardson
<bruce.richardson at intel.com> wrote:
>
> On Fri, Apr 09, 2021 at 02:41:11PM +0100, Bruce Richardson wrote:
> > On Fri, Apr 09, 2021 at 03:27:17PM +0200, David Marchand wrote:
> > > On Wed, Jan 27, 2021 at 6:43 PM Bruce Richardson
> > > <bruce.richardson at intel.com> wrote:
> > > >
> > > > While having the ability to run a test based off the DPDK_TEST environment
> > > > variable is useful, it's often easier to specify the test name as a
> > > > commandline parameter to a test binary. This also allows the test runs to
> > > > be saved as part of the shell cmdline history.
> > >
> > > I don't get the argument about history:
> > >
> > > $ history |grep DPDK_TEST
> > > 10615  2021-03-24 10:42:11 ninja-build -C build -j4 &&
> > > DPDK_TEST=logs_autotest ./build/app/test/dpdk-test --no-huge -m 512
> > > --log-level=lib.eal:debug
> > > 10636  2021-03-24 10:51:09 ninja-build -C build -j4 &&
> > > DPDK_TEST=logs_autotest ./build/app/test/dpdk-test --no-huge -m 512
> > > --log-level=lib.eal:debug
> > > 10653  2021-03-24 11:17:01 ninja-build -C build -j4 &&
> > > DPDK_TEST=kvargs_autotest ./build/app/test/dpdk-test --no-huge -m 512
> > > --log-level=lib.eal:debug
> > > 10794  2021-03-25 18:37:48 history |grep DPDK_TEST
> > >
> >
> > Sure, if you always specify the test name explicitly for each command,
> > rather than running the one test multiple times having set it separately in
> > the environment.
> > Overall, though I take the point that from a history saving point of view
> > it's a minor saving.
> >
> > >
> > > >
> > > > This patch adds support for checking all parameters after the EAL ones, and
> > > > running all valid autotests requested - either from DPDK_TEST or on the
> > > > commandline. This also allows multiple tests to be run in a single
> > > > automated session, which is useful for working with components which have
> > > > multiple test suites.
> > >
> > > The same could be achieved splitting DPDK_TEST content with spaces,
> > > since test names don't contain one.
> > >
> >
> > Yep, that's a useful enhancement too, but I still thing it's better to just
> > have the list of tests appended to the test binary command rather than have
> > to be worrying about properly quoting a specific environment variable at
> > the start of each command.
> >
> Ping on this.

I have been using the DPDK_TEST= method for quite some time.
But I don't think I ever had a need to run multiple tests since I
usually track regressions or hard to reproduce failures.
When developing a new test, idem, I used DPDK_TEST=.


>
> Other instances where using cmdline is preferred over environment
>
> * Calling tests using sudo.

I guess calling a shell with the same command would work, something like:
$ sudo sh -c 'DPDK_TEST=debug_autotest ..../build/app/test/dpdk-test
--no-huge -m 512'

> * Calling tests as an execute action when doing a git rebase

I am pretty sure I used DPDK_TEST= in rebase execute actions in the past too.
This probably works too:
$ git rebase -i HEAD^ -x 'DPDK_TEST=debug_autotest
build/app/test/dpdk-test --no-huge -m 512'


>
> Yes, again in both cases, other workarounds are generally available (e.g.
> sudo -E, and exporting to environemtn before rebase), but also generally
> less convenient.

You find it less convenient when you want to run multiple tests.
I don't have that use case but I don't mind taking it.
Can you simply adjust the commitlog, or propose a new wording I can
take when applying?


Thanks.

-- 
David Marchand