<html>
<head>
<meta http-equiv="Content-Type" content="text/html; charset=UTF-8">
</head>
<body>
<div class="moz-cite-prefix">Hi Adam,<br>
<br>
I've uploaded fresh testing results to ts-factory.io [1] to be on
the same page.<br>
<br>
I think I know why your and mine results on Intel 710 series NICs
differ so much. Testing results expectations database
(dpdk-ethdev-ts/trc/*) is filled in in terms of TRC tags. I.e.
expectations depends on TRC tags discovered by helper scripts when
testing is started. These tags identify various aspects of what is
tested. Ideally expectations should be written in terms of root
cause of the expected behaviour. If it is a driver expectations,
driver tag should be used. If it is HW limitation, tags with PCI
IDs should be used. However, it is not always easy to classify it
correctly if you're not involved in driver development. So, in
order case expectations for 710's Intel are filled in in terms of
PCI IDs. I guess PCI ID differ in your case and that's why
expectations filled in for my NIC do not apply to your runs.<br>
<br>
Just try to add the following option when you run on your 710's
Intel in order to mimic mine and see if it helps to achieve better
pass rate.<br>
--trc-tag=pci-8086-1572<br>
<br>
BTW, fresh TE tag <span class="text-[0.625rem] font-medium
leading-[1.125rem]">v1.21.0 has improved algorithm to choose
tests for --tester-dial option. It should have better coverage
now.</span><br>
<br>
Andrew.<br>
<br>
[1]
<a class="moz-txt-link-freetext" href="https://ts-factory.io/bublik/v2/runs?startDate=2023-09-16&finishDate=2023-09-16&runData=&runDataExpr=&page=1">https://ts-factory.io/bublik/v2/runs?startDate=2023-09-16&finishDate=2023-09-16&runData=&runDataExpr=&page=1</a><br>
<br>
On 9/13/23 18:45, Andrew Rybchenko wrote:<br>
</div>
<blockquote type="cite"
cite="mid:ad89983f-70e4-2158-2104-7601d5c1528f@oktetlabs.ru">
<meta http-equiv="Content-Type" content="text/html; charset=UTF-8">
<div class="moz-cite-prefix">Hi Adam,<br>
<br>
I've pushed new TE tag v1.20.0 which supported a new
command-line option --tester-dial=NUM where NUM is from 0 to
100. it allows to choose percentage of tests to run. If you want
stable set, you should pass --tester-random-seed=0 (or other
integer). It is the first sketch and we have plans to improve
it, but feedback would be welcome.<br>
<br>
> Is it needed on the tester?<br>
<br>
It is hard to say if it is strictly required for simple tests.
However, it is better to update Tester as well, since
performance tests run DPDK on Tester as well.<br>
<br>
> Are there any other manual setup steps for these devices
that I might be missing?<br>
<br>
I don't remember anything else.<br>
<br>
I think it is better to get down to details and take a look at
logs. I'm ready to help with it and explain what's happening
there. May be it will help to understand if it is a problem with
setup/configuration.<br>
<br>
Text logs are not very convenient. Ideally logs should be
imported to bublik, however, manual runs do not provide all
required artifacts right now (Jenkins jobs generate all required
artifacts).<br>
Other option is 'tmp_raw_log' file (should be packed to make it
smaller) which could be converted to various log formats.<br>
Would it be OK for you if I import your logs to bublik at
ts-factory.io? Or is it a problem that it is publicly available?<br>
Would it help if we add authentication and access control there?<br>
<br>
Andrew.<br>
<br>
On 9/8/23 17:57, Adam Hassick wrote:<br>
</div>
<blockquote type="cite"
cite="mid:CAC-YWqjOhiYXkBAK5pLnMqUNkB-z4Y=JAUMeN3fc6AMfjOzPrg@mail.gmail.com">
<meta http-equiv="content-type" content="text/html;
charset=UTF-8">
<div dir="ltr">
<div>
<div>
<div>
<div>
<div>Hi Andrew,<br>
<br>
</div>
I have a couple questions about needed setup of the
NICs for the ethdev test suite.<br>
<br>
</div>
Our MCX5s and XL710s are failing the checkup tests. The
pass rate appears to be much worse on the XL710s (40 of
73 tests failed, 3 passed unexpectedly).<br>
<br>
</div>
For the XL710s, I've updated the driver and NVM versions
to match the minimum supported versions in the
compatibility matrix found on the DPDK documentation. This
did not change the failure rate much.<br>
</div>
For the MCX5s, I've installed the latest LTS version of the
OFED bifurcated driver on the DUT. Is it needed on the
tester?<br>
<br>
</div>
Are there any other manual setup steps for these devices that
I might be missing?<br>
<div>
<div>
<div>
<div>
<div>
<div>
<div><br>
</div>
<div>Thanks,<br>
</div>
<div>Adam<br>
</div>
</div>
</div>
</div>
</div>
</div>
</div>
</div>
<br>
<div class="gmail_quote">
<div dir="ltr" class="gmail_attr">On Wed, Sep 6, 2023 at
11:00 AM Adam Hassick <<a
href="mailto:ahassick@iol.unh.edu" moz-do-not-send="true"
class="moz-txt-link-freetext">ahassick@iol.unh.edu</a>>
wrote:<br>
</div>
<blockquote class="gmail_quote" style="margin:0px 0px 0px
0.8ex;border-left:1px solid
rgb(204,204,204);padding-left:1ex">
<div dir="ltr">
<div>
<div>
<div>
<div>
<div>Hi Andrew,<br>
<br>
</div>
<div>Yes, I copied the X710 configs to set up
XL710 configs. I changed the environment
variable names from the X710 suffix to XL710
suffix in the script, and forgot to change them
in the corresponding environment file.<br>
</div>
</div>
That fixed the issue.<br>
<br>
</div>
I got the checkup tests working on the XL710 now. Most
of them are failing, which leads me to believe this is
an issue with our testbed. Based on the DPDK
documentation for i40e, the firmware and driver
versions are much older than what DPDK 22.11 LTS and
main prefer, so I'll try updating those.<br>
<br>
</div>
For now I'm working on getting the XL710 checkup tests
passing, and will pick up getting the E810 configured
properly next. I'll let you know if I run into any more
issues in relation to the test engine.<br>
<br>
</div>
<div>Thanks,<br>
</div>
<div>Adam<br>
</div>
</div>
<br>
<div class="gmail_quote">
<div dir="ltr" class="gmail_attr">On Wed, Sep 6, 2023 at
7:36 AM Andrew Rybchenko <<a
href="mailto:andrew.rybchenko@oktetlabs.ru"
target="_blank" moz-do-not-send="true"
class="moz-txt-link-freetext">andrew.rybchenko@oktetlabs.ru</a>>
wrote:<br>
</div>
<blockquote class="gmail_quote" style="margin:0px 0px 0px
0.8ex;border-left:1px solid
rgb(204,204,204);padding-left:1ex">
<div>
<div>Hi Adam,<br>
<br>
On 9/5/23 18:01, Adam Hassick wrote:<br>
</div>
<blockquote type="cite">
<div dir="ltr">
<div>
<div>
<div>
<div>
<div>
<div>Hi Andrew,<br>
<br>
</div>
The compilation warning issue is now
resolved. Again, thank you guys for
fixing this for us. I can run the tests
on the Mellanox CX5s again, however I'm
running into a couple new issues with
running the prologues on the Intel
cards.<br>
<br>
</div>
When running testing on the Intel XL710s,
I see this error appear in the log:<br>
<br>
<blockquote class="gmail_quote"
style="margin:0px 0px 0px
0.8ex;border-left:1px solid
rgb(204,204,204);padding-left:1ex">ERROR
prologue Environment LIB 14:16:13.650<br>
Too few networks in available
configuration (0) in comparison with
required (1)<br>
</blockquote>
<br>
</div>
This seems like a trivial configuration
error, perhaps this is something I need to
set up in ts-rigs. I briefly searched
through the examples there and didn't see
any mention of how to set up a network.<br>
</div>
<div>I will attach this log just in case you
need more information.<br>
</div>
</div>
</div>
</div>
</blockquote>
<br>
Unfortunately logs are insufficient to understand it.
I've pushed new tag to TE v1.19.0 which add log
message with TE_* environment variables.<br>
Most likely something is wrong with variables which
are used as conditions when available networks are
defined in ts-conf/cs/inc.net_cfg_pci_fns.yml:<br>
TE_PCI_INSTANCE_IUT_TST1<br>
TE_PCI_INSTANCE_IUT_TST1a<br>
TE_PCI_INSTANCE_TST1a_IUT<br>
TE_PCI_INSTANCE_TST1_IUT<br>
My guess it that you change naming a bit, but script
like ts-rigs-sample/scripts/iut.h1-x710 is not
included or not updated.<br>
<br>
<blockquote type="cite">
<div dir="ltr">
<div>
<div>There is a different error when running on
the Intel E810s. It appears to me like it
starts DPDK, does some configuration inside
DPDK and on the device, and then fails to
bring the device back up. Since this error
seems very non-trivial, I will also attach
this log.<br>
</div>
</div>
</div>
</blockquote>
<br>
This one is a bit simpler. Few lines after the first
ERROR in log I see the following:<br>
WARN RCF DPDK 13:06:00.144<br>
ice_program_hw_rx_queue(): currently package doesn't
support RXDID (22)<br>
ice_rx_queue_start(): fail to program RX queue 0<br>
ice_dev_start(): fail to start Rx queue 0<br>
Device with port_id=0 already stopped<br>
<br>
It is stdout/stderr from test agent which runs DPDK.
Same logs in plain format are available in ta.DPDK
file.<br>
I'm not an expert here, but I vaguely remember that
E810 requires correct firmware and DDP to be loaded.<br>
There is some information in
dpdk/doc/guides/nics/ice.rst.<br>
<br>
You can try to add --dev-args=safe-mode-support=1
command-line option described there.<br>
<br>
Hope it helps,<br>
Andrew.<br>
<br>
<blockquote type="cite">
<div dir="ltr">
<div>
<div><br>
</div>
Thanks,<br>
</div>
Adam<br>
</div>
<br>
<div class="gmail_quote">
<div dir="ltr" class="gmail_attr">On Fri, Sep 1,
2023 at 3:59 AM Andrew Rybchenko <<a
href="mailto:andrew.rybchenko@oktetlabs.ru"
target="_blank" moz-do-not-send="true"
class="moz-txt-link-freetext">andrew.rybchenko@oktetlabs.ru</a>>
wrote:<br>
</div>
<blockquote class="gmail_quote" style="margin:0px
0px 0px 0.8ex;border-left:1px solid
rgb(204,204,204);padding-left:1ex">
<div>
<div>Hi Adam,<br>
<br>
On 8/31/23 22:38, Adam Hassick wrote:<br>
</div>
<blockquote type="cite">
<div dir="ltr">
<div>Hi Andrew,<br>
</div>
<div><br>
I have one additional question as well:
Does the test engine support running
tests on two ARMv8 test agents?</div>
<div><br>
</div>
<div>
<blockquote class="gmail_quote"
style="margin:0px 0px 0px
0.8ex;border-left:1px solid
rgb(204,204,204);padding-left:1ex">1.
We'll sort out warnings this week.
Thanks for heads up.<br>
</blockquote>
<div><br>
</div>
<div>Great. Let me know when that's
fixed.</div>
</div>
</div>
</blockquote>
<br>
Done. We also fixed a number of warnings in
TE.<br>
Also we fixed root test package name to be
consistent with the repository name.<br>
<br>
<blockquote type="cite">
<div dir="ltr">
<blockquote class="gmail_quote"
style="margin:0px 0px 0px
0.8ex;border-left:1px solid
rgb(204,204,204);padding-left:1ex">
<div>Support for old LTS branches was
dropped some time ago, but in the
future it is definitely possible to
keep it for new LTS branches. I think
22.11 is supported, but I'm not sure
about older LTS releases.</div>
</blockquote>
<div><br>
</div>
<div>Good to know.<br>
<div> <br>
<blockquote class="gmail_quote"
style="margin:0px 0px 0px
0.8ex;border-left:1px solid
rgb(204,204,204);padding-left:1ex">
2. You can add command-line option
--sanity to run tests marked with
TEST_HARNESS_SANITY requirement (see
dpdk-ethdev-ts/scripts/run.sh and
grep TEST_HARNESS_SANITY
dpdk-ethdev-ts to see which tests
are marked). Yes, there is a space
for terminology improvement here.
We'll do it.<br>
</blockquote>
</div>
</div>
</div>
</blockquote>
<br>
Done. Now it is called --checkup.<br>
<br>
<blockquote type="cite">
<div dir="ltr">
<div>
<div>
<blockquote class="gmail_quote"
style="margin:0px 0px 0px
0.8ex;border-left:1px solid
rgb(204,204,204);padding-left:1ex"><br>
Also it takes a lot of time because
of failures and tests which wait for
some timeout.<br>
</blockquote>
</div>
<div><br>
</div>
<div>That makes sense to me. We'll use
the time to complete tests on virtio
or the Intel devices as a reference
for how long the tests really take to
complete.<br>
</div>
<div>We will explore the possibility of
periodically running the sanity tests
for patches.<br>
</div>
</div>
</div>
</blockquote>
<br>
I'll double-check and let you know how long
entire TS runs on Intel X710, E810, Mellanox
CX5 and virtio net. Just to ensure that time
observed in your case looks the same.<br>
<br>
<blockquote type="cite">
<div dir="ltr">
<div>
<div> <br>
<blockquote class="gmail_quote"
style="margin:0px 0px 0px
0.8ex;border-left:1px solid
rgb(204,204,204);padding-left:1ex">
The test harness can provide
coverage reports based on gcov, but
I'm not sure what you mean by a
"dial" to control test coverage.
Provided reports are rather for
human to analyze.<br>
</blockquote>
</div>
<div><br>
</div>
<div>The general idea is to have some
kind of parameter on the test suite,
which could be an integer ranging from
zero to ten, that controls how many
tests are run based on how important
the test is.<br>
<br>
</div>
<div>Similar to how some command line
interfaces provide a verbosity level
parameter (some number of "-v"
arguments) to control the importance
of the information in the log.<br>
</div>
The verbosity level zero only prints
very important log messages, while ten
prints everything.<br>
</div>
<div><br>
In much the same manner as above, this
"dial" parameter controls what tests are
run and with what parameters based on
how important those tests and test
parameter combinations are.<br>
Coverage Level zero tells the suite to
run a very basic set of important tests,
with minimal parameterization. This mode
would take only ~5-10 minutes to run.<br>
In contrast, Coverage Level ten includes
all the edge cases, every combination of
test parameters, everything the test
suite can do, which takes the normal
several hours to run.<br>
The values 1 - 9 are between those two
extremes, allowing the user to get a
gradient of test coverage in the results
and to limit the running time.<br>
<br>
</div>
Then we could, for example, run the
"run.sh" with a level of 2 or 3 for
incoming patches that need quick results,
and with a level of 10 for the less often
run periodic tests performed on main or
LTS branches.<br>
</div>
</blockquote>
<br>
Understood now. Thanks a lot for the idea.
We'll discuss it and come back.<br>
<br>
<blockquote type="cite">
<div dir="ltr">
<div>
<div>
<div> </div>
<blockquote class="gmail_quote"
style="margin:0px 0px 0px
0.8ex;border-left:1px solid
rgb(204,204,204);padding-left:1ex">
<div>
<div> 3. Yes, really many tests on
Mellanox CX5 NICs report
unexpected testing results.
Unfortunately it is time
consuming to fill in
expectations database since it
is necessary to analyze testing
results and classify if it is a
bug or just acceptable behaviour
aspect.<br>
<br>
Bublik allows to compare results
of two runs. It is useful for
human, but still not good for
automation.<br>
<br>
I have local patch for mlx5
driver which reports Tx ring
size maximum. It makes pass rate
higher. It is a problem for test
harness that mlx5 does not
report limits right now.<br>
<br>
Pass rate on Intel X710 is about
92% on my test rig. Pass rate on
virtio net is 99% right now and
could be done 100% easily (just
one thing to fix in
expectations).<br>
<br>
I think logs storage setup is
essential for logs analysis. Of
course, you can request HTML
logs when you run tests
(--log-html=html) or generate
after run using
dpdk-ethdev-ts/scripts/html-log.sh
and open index.html in a
browser, but logs storage makes
it more convenient.<br>
</div>
</div>
</blockquote>
<div><br>
We are interested in setting up
Bublik, potentially as an
externally-facing component, once we
have our process of running the test
suite stabilized.</div>
<div>Once we are able to run the test
suite again, I'll see what the pass
rate is on our other hardware.<br>
Good to know that it isn't an issue
with our dev testbed causing the
high fail rate.</div>
</div>
<div>
<div><br>
</div>
<div>For Intel hardware, we have an
XL710 and an Intel E810-C in our
development testbed. Although they
are slightly different devices,
ideally the pass rate will be
identical or similar. I have yet to
set up a VM pair for virtio, but we
will soon.<br>
</div>
<div><br>
<blockquote class="gmail_quote"
style="margin:0px 0px 0px
0.8ex;border-left:1px solid
rgb(204,204,204);padding-left:1ex">
Latest version of test-environment
has examples of our CGI scripts
which we use for log storage (see
tools/log_server/README.md).<br>
<br>
Also all bits for Jenkins setup
are available. See
dpdk-ethdev-ts/jenkins/README.md
and examples of jenkins files in
ts-rigs-sample.<br>
</blockquote>
</div>
<div><br>
</div>
<div>Jenkins integration, setting up
production rig configurations, and
permanent log storage will be our
next steps once I am able to run the
tests again.<br>
</div>
<div>Unless there is an easy way to
have meson not pass "-Werror" into
GCC. Then I would be able to run the
test suite.<br>
</div>
</div>
</div>
</div>
</blockquote>
<br>
Hopefully it is resolved now.<br>
<br>
I thought a bit more about your usecase for
Jenkins. I'm not 100% sure that existing
pipelines are convenient for your usecase.<br>
Fill free to ask questions when you are on it.<br>
<br>
Thanks,<br>
Andrew.<br>
<br>
<blockquote type="cite">
<div dir="ltr">
<div>
<div>
<div><br>
</div>
<div>Thanks,<br>
</div>
<div>Adam<br>
</div>
<div><br>
</div>
<div> </div>
<blockquote class="gmail_quote"
style="margin:0px 0px 0px
0.8ex;border-left:1px solid
rgb(204,204,204);padding-left:1ex">
<div>
<div> <br>
On 8/29/23 17:02, Adam Hassick
wrote:<br>
</div>
<blockquote type="cite">
<div dir="ltr">
<div>
<div>
<div>Hi Andrew,<br>
<br>
</div>
That fix seems to have
resolved the issue, thanks
for the quick turnaround
time on that patch.<br>
</div>
<div>Now that we have the
RCF timeout issue
resolved, there are a few
other questions and issues
that we have about the
tests themselves.</div>
<br>
</div>
<div>1. The test suite fails
to build with a couple
warnings.<br>
</div>
<div><br>
</div>
<div>Below is the stderr log
from compilation:<br>
</div>
<br>
<blockquote
class="gmail_quote"
style="margin:0px 0px 0px
0.8ex;border-left:1px solid
rgb(204,204,204);padding-left:1ex">FAILED: <a
href="mailto:lib/76b5a35@@ts_dpdk_pmd@sta/dpdk_pmd_ts.c.o"
target="_blank"
moz-do-not-send="true"
class="moz-txt-link-freetext">lib/76b5a35@@ts_dpdk_pmd@sta/dpdk_pmd_ts.c.o</a><br>
cc
-Ilib/76b5a35@@ts_dpdk_pmd@sta
-Ilib -I../../lib
-I/opt/tsf/dpdk-ethdev-ts/ts/inst/default/include
-fdiagnostics-color=always
-pipe -D_FILE_OFFSET_BITS=64
-Wall -Winvalid-pch -Werror
-g -D_GNU_SOURCE -O0 -ggdb
-Wall -W -fPIC -MD -MQ '<a
href="mailto:lib/76b5a35@@ts_dpdk_pmd@sta/dpdk_pmd_ts.c.o"
target="_blank"
moz-do-not-send="true"
class="moz-txt-link-freetext">lib/76b5a35@@ts_dpdk_pmd@sta/dpdk_pmd_ts.c.o</a>'
-MF '<a
href="mailto:lib/76b5a35@@ts_dpdk_pmd@sta/dpdk_pmd_ts.c.o.d"
target="_blank"
moz-do-not-send="true"
class="moz-txt-link-freetext">lib/76b5a35@@ts_dpdk_pmd@sta/dpdk_pmd_ts.c.o.d</a>'
-o '<a
href="mailto:lib/76b5a35@@ts_dpdk_pmd@sta/dpdk_pmd_ts.c.o"
target="_blank"
moz-do-not-send="true"
class="moz-txt-link-freetext">lib/76b5a35@@ts_dpdk_pmd@sta/dpdk_pmd_ts.c.o</a>'
-c ../../lib/dpdk_pmd_ts.c<br>
../../lib/dpdk_pmd_ts.c: In
function
‘test_create_traffic_generator_params’:<br>
../../lib/dpdk_pmd_ts.c:5577:5: error: format not a string literal and
no format arguments
[-Werror=format-security]<br>
5577 | rc =
te_kvpair_add(result, buf,
mode);<br>
| ^~<br>
cc1: all warnings being
treated as errors<br>
ninja: build stopped:
subcommand failed.<br>
ninja: Entering directory
`.'<br>
FAILED: <a
href="mailto:lib/76b5a35@@ts_dpdk_pmd@sta/dpdk_pmd_ts.c.o"
target="_blank"
moz-do-not-send="true"
class="moz-txt-link-freetext">lib/76b5a35@@ts_dpdk_pmd@sta/dpdk_pmd_ts.c.o</a><br>
cc
-Ilib/76b5a35@@ts_dpdk_pmd@sta
-Ilib -I../../lib
-I/opt/tsf/dpdk-ethdev-ts/ts/inst/default/include
-fdiagnostics-color=always
-pipe -D_FILE_OFFSET_BITS=64
-Wall -Winvalid-pch -Werror
-g -D_GNU_SOURCE -O0 -ggdb
-Wall -W -fPIC -MD -MQ '<a
href="mailto:lib/76b5a35@@ts_dpdk_pmd@sta/dpdk_pmd_ts.c.o"
target="_blank"
moz-do-not-send="true"
class="moz-txt-link-freetext">lib/76b5a35@@ts_dpdk_pmd@sta/dpdk_pmd_ts.c.o</a>'
-MF '<a
href="mailto:lib/76b5a35@@ts_dpdk_pmd@sta/dpdk_pmd_ts.c.o.d"
target="_blank"
moz-do-not-send="true"
class="moz-txt-link-freetext">lib/76b5a35@@ts_dpdk_pmd@sta/dpdk_pmd_ts.c.o.d</a>'
-o '<a
href="mailto:lib/76b5a35@@ts_dpdk_pmd@sta/dpdk_pmd_ts.c.o"
target="_blank"
moz-do-not-send="true"
class="moz-txt-link-freetext">lib/76b5a35@@ts_dpdk_pmd@sta/dpdk_pmd_ts.c.o</a>'
-c ../../lib/dpdk_pmd_ts.c<br>
../../lib/dpdk_pmd_ts.c: In
function
‘test_create_traffic_generator_params’:<br>
../../lib/dpdk_pmd_ts.c:5577:5: error: format not a string literal and
no format arguments
[-Werror=format-security]<br>
5577 | rc =
te_kvpair_add(result, buf,
mode);<br>
| ^~<br>
cc1: all warnings being
treated as errors<br>
</blockquote>
<div>
<div>
<div><br>
</div>
<div>This error wasn't
occurring last week,
which was the last time
I ran the tests.<br>
</div>
<div>The TE host and the
DUT have GCC v9.4.0
installed, and the
tester has GCC v11.4.0
installed, if this
information is helpful.<br>
</div>
<div><br>
</div>
<div>2. On the Mellanox
CX5s, there are over
6,000 tests run, which
collectively take around
9 hours. Is it possible,
and would it make sense,
to lower the test
coverage and have the
test suite run faster?<br>
<br>
</div>
<div>For some context, we
run immediate testing on
incoming patches for
DPDK main and
development branches, as
well as periodic test
runs on the main,
stable, and LTS
branches.<br>
</div>
<div>For us to consider
including this test
suite as part of our
immediate testing on
patches, we would have
to reduce the test
coverage to the most
important tests.<br>
This is primarily to
reduce the testing time
to, for example, less
than 30 minutes. Testing
on patches can't take
too long because the lab
can receive numerous
patches each day, which
each require individual
testing runs.<br>
<br>
</div>
<div>At what frequency we
run these tests, and on
what, still needs to be
discussed with the DPDK
community, but it would
be nice to know if the
test suite had a "dial"
to control the testing
coverage.<br>
</div>
<div><br>
</div>
<div>3. We see a lot of
test failures on our
Mellanox CX5 NICs.
Around 2,300 of ~6,600
tests passed. Is there
anything we can do to
diagnose these test
failures?<br>
</div>
<div><br>
</div>
<div>Thanks,<br>
</div>
<div>Adam<br>
</div>
<div><br>
</div>
</div>
</div>
</div>
<br>
<div class="gmail_quote">
<div dir="ltr"
class="gmail_attr">On Tue,
Aug 29, 2023 at 8:07 AM
Andrew Rybchenko <<a
href="mailto:andrew.rybchenko@oktetlabs.ru"
target="_blank"
moz-do-not-send="true"
class="moz-txt-link-freetext">andrew.rybchenko@oktetlabs.ru</a>>
wrote:<br>
</div>
<blockquote
class="gmail_quote"
style="margin:0px 0px 0px
0.8ex;border-left:1px solid
rgb(204,204,204);padding-left:1ex">
<div>
<div>Hi Adam,<br>
<br>
I've pushed the fix in
main branch and a new
tag v1.18.1. It should
solve the problem with
IPv6 address from DNS.<br>
<br>
Andrew.<br>
<br>
On 8/29/23 00:05, Andrew
Rybchenko wrote:<br>
</div>
<blockquote type="cite">
<div>Hi Adam,<br>
<br>
> Does the test
engine prefer to use
IPv6 over IPv4 for
initiating the RCF
connection to the test
bed hosts? And if so,
is there a way to
force it to use IPv4?<br>
<br>
Brilliant idea. If DNS
returns both IPv4 and
IPv6 addresses in your
case, I guess it is
the root cause of the
problem.<br>
Of course, it is TE
problem since I see
really weird code in
lib/comm_net_engine/comm_net_engine.c
line 135.<br>
<br>
I've pushed fix to the
branch
user/arybchik/fix_ipv4_only
in
ts-factory/test-environment
repository. Please,
try.<br>
<br>
It is late night fix
with minimal testing
and no review. I'll
pass it through review
process tomorrow and<br>
hopefully it will be
released in one-two
days.<br>
<br>
Andrew.<br>
<br>
On 8/28/23 18:02, Adam
Hassick wrote:<br>
</div>
<blockquote type="cite">
<div dir="ltr">
<div>
<div>
<div>Hi Andrew,<br>
<br>
</div>
We have yet to
notice a
distinct pattern
with the
failures.
Sometimes, the
RCF will start
and connect
without issue a
few times in a
row before
failing to
connect again.
Once the issue
begins to occur,
neither
rebooting all of
the hosts (test
engine VM,
tester, IUT) or
deleting all of
the build
directories
(suites, agents,
inst) and
rebooting the
hosts afterward
resolves the
issue. When it
begins working
again seems very
arbitrary to us.<br>
<br>
</div>
<div>I do usually
try to terminate
the test engine
with Ctrl+C, but
when it hangs
while trying to
start RCF, that
does not work.<br>
</div>
<div><br>
</div>
<div>Does the test
engine prefer to
use IPv6 over
IPv4 for
initiating the
RCF connection
to the test bed
hosts? And if
so, is there a
way to force it
to use IPv4?<br>
<br>
</div>
<div> - Adam<br>
</div>
</div>
</div>
<br>
<div
class="gmail_quote">
<div dir="ltr"
class="gmail_attr">On
Fri, Aug 25, 2023
at 1:35 PM Andrew
Rybchenko <<a
href="mailto:andrew.rybchenko@oktetlabs.ru"
target="_blank"
moz-do-not-send="true" class="moz-txt-link-freetext">andrew.rybchenko@oktetlabs.ru</a>>
wrote:<br>
</div>
<blockquote
class="gmail_quote"
style="margin:0px
0px 0px
0.8ex;border-left:1px
solid
rgb(204,204,204);padding-left:1ex">
<div>
<div>> I'll
double-check
test engine on
Ubuntu 20.04
and Ubuntu
22.04.<br>
<br>
Done. It works
fine for me
without any
issues.<br>
<br>
Have you
noticed any
pattern when
it works or
does not work?<br>
May be it is a
problem of not
clean state
after
termination?<br>
Does it work
fine the first
time after
DUTs reboot?<br>
How do you
terminate
testing? It
should be done
using Ctrl+C
in terminal
where you
execute run.sh
command.<br>
In this case
it should
shutdown
gracefully and
close all test
agents and
engine
applications.<br>
<br>
(I'm trying to
understand why
you've seen
many test
agent
processes. It
should not
happen.)<br>
<br>
Andrew.<br>
<br>
On 8/25/23
17:41, Andrew
Rybchenko
wrote:<br>
</div>
<blockquote
type="cite">
<div>On
8/25/23 17:06,
Adam Hassick
wrote:<br>
</div>
<blockquote
type="cite">
<div dir="ltr">
<div>
<div>Hi
Andrew,<br>
<br>
</div>
Two of our
systems (the
Test Engine
runner and the
DUT host) are
running Ubuntu
20.04 LTS,
however this
morning I
noticed that
the tester
system (the
one having
issues) is
running Ubuntu
22.04 LTS.<br>
</div>
<div>This
could be the
source of the
problem. I
encountered a
dependency
issue trying
to run the
Test Engine on
22.04 LTS, so
I downgraded
the system.
Since the
tester is also
the host
having
connection
issues, I will
try
downgrading
that system to
20.04, and see
if that
changes
anything.<br>
</div>
</div>
</blockquote>
<br>
Unlikely, but
who knows. We
run tests
(DUTs) on
Ubuntu 20.04,
Ubuntu 22.04,
Ubuntu 22.10,
Ubuntu 23.04,
Debian 11 and
Fedora 38
every night.<br>
Right now
Debian 11 is
used for test
engine in
nightly
regressions.<br>
<br>
I'll
double-check
test engine on
Ubuntu 20.04
and Ubuntu
22.04.<br>
<br>
<blockquote
type="cite">
<div dir="ltr">
<div>I did try
passing in the
"--vg-rcf"
argument to
the run.sh
script of the
test suite
after
installing
valgrind, but
there was no
additional
output that I
saw.<br>
</div>
</div>
</blockquote>
<br>
Sorry, I
should
valgrind
output should
be in
valgrind.te_rcf
(direction
where you run
test engine).<br>
<br>
<blockquote
type="cite">
<div dir="ltr">
<div><br>
</div>
<div>I will
try pulling in
the changes
you've pushed
up, and will
see if that
fixes
anything.<br>
<br>
</div>
<div>Thanks,<br>
</div>
<div>Adam<br>
</div>
</div>
<br>
<div
class="gmail_quote">
<div dir="ltr"
class="gmail_attr">On Fri, Aug 25, 2023 at 9:57 AM Andrew Rybchenko <<a
href="mailto:andrew.rybchenko@oktetlabs.ru" target="_blank"
moz-do-not-send="true"
class="moz-txt-link-freetext">andrew.rybchenko@oktetlabs.ru</a>>
wrote:<br>
</div>
<blockquote
class="gmail_quote"
style="margin:0px 0px 0px 0.8ex;border-left:1px solid
rgb(204,204,204);padding-left:1ex">
<div>
<div>Hello
Adam, <br>
<br>
On 8/24/23
23:54, Andrew
Rybchenko
wrote:<br>
</div>
<blockquote
type="cite">I'd
like to try to
repeat the
problem
locally. Which
Linux distro
is running on
test engine
and agents? <br>
<br>
In fact I know
one problem
with Debian 12
and Fedora 38
and we have <br>
patch in
review to fix
it, however,
the behaviour
is different
in <br>
this case, so
it is unlike
the same
problem. <br>
</blockquote>
<br>
I've just
published a
new tag which
fixes known
test engine
side problems
on Debian 12
and Fedora 38.<br>
<br>
<blockquote
type="cite"> <br>
One more idea
is to install
valgrind on
the test
engine host
and <br>
run with
option
--vg-rcf to
check if
something
weird is
happening. <br>
<br>
What I don't
understand
right now is
why I see just
one failed
attempt <br>
to connect in
your log.txt
and then
Logger
shutdown after
9 minutes. <br>
<br>
Andrew. <br>
<br>
On 8/24/23
23:29, Adam
Hassick wrote:
<br>
<blockquote
type="cite"> >
Is there any
firewall in
the network or
on test hosts
which could
block incoming
TCP connection
to the port
23571 <a
href="http://iol-dts-tester.dpdklab.iol.unh.edu:23571"
target="_blank" moz-do-not-send="true"><http://iol-dts-tester.dpdklab.iol.unh.edu:23571></a>
from the host
where you run
test engine? <br>
<br>
Our test
engine host
and the
testbed are on
the same
subnet. The
connection
does work
sometimes. <br>
<br>
> If
behaviour the
same on the
next try and
you see that
test agent is
kept running,
could you
check using <br>
> <br>
> #
netstat -tnlp
<br>
> <br>
> that
Test Agent is
listening on
the port and
try to
establish TCP
connection
from test
agent using <br>
> <br>
> $ telnet
<a
href="http://iol-dts-tester.dpdklab.iol.unh.edu"
target="_blank" moz-do-not-send="true">iol-dts-tester.dpdklab.iol.unh.edu</a>
<a
href="http://iol-dts-tester.dpdklab.iol.unh.edu:23571"
target="_blank" moz-do-not-send="true"><http://iol-dts-tester.dpdklab.iol.unh.edu:23571></a>
23571 <a
href="http://iol-dts-tester.dpdklab.iol.unh.edu:23571"
target="_blank" moz-do-not-send="true"><http://iol-dts-tester.dpdklab.iol.unh.edu:23571></a>
<br>
> <br>
> and
check if TCP
connection
could be
established. <br>
<br>
I was able to
replicate the
same behavior
again, where
it hangs while
RCF is trying
to start. <br>
Running this
command, I see
this in the
output: <br>
<br>
tcp 0
0 <a
href="http://0.0.0.0:23571"
target="_blank" moz-do-not-send="true">0.0.0.0:23571</a> <a
href="http://0.0.0.0:23571"
target="_blank" moz-do-not-send="true"><http://0.0.0.0:23571></a>
0.0.0.0:*
LISTEN
18599/ta <br>
<br>
So it seems
like it is
listening on
the correct
port. <br>
Additionally,
I was able to
connect to the
Tester machine
from our Test
Engine host
using telnet.
It printed the
PID of the
process once
the connection
was opened. <br>
<br>
I tried
running the
"ta"
application
manually on
the command
line, and it
didn't print
anything at
all. <br>
Maybe the
issue is
something on
the Test
Engine side. <br>
<br>
On Thu, Aug
24, 2023 at
2:35 PM Andrew
Rybchenko <<a
href="mailto:andrew.rybchenko@oktetlabs.ru" target="_blank"
moz-do-not-send="true"
class="moz-txt-link-freetext">andrew.rybchenko@oktetlabs.ru</a> <a
href="mailto:andrew.rybchenko@oktetlabs.ru"
target="_blank" moz-do-not-send="true"><mailto:andrew.rybchenko@oktetlabs.ru></a>>
wrote: <br>
<br>
Hi Adam, <br>
<br>
> On
the tester
host (which
appears to be
the Peer
agent), there
<br>
are four
processes that
I see running,
which look
like the test
<br>
agent
processes. <br>
<br>
Before the
next try I'd
recommend to
kill these
processes. <br>
<br>
Is there
any firewall
in the network
or on test
hosts which
could <br>
block
incoming TCP
connection to
the port 23571
<br>
<a
href="http://iol-dts-tester.dpdklab.iol.unh.edu:23571"
target="_blank" moz-do-not-send="true"><http://iol-dts-tester.dpdklab.iol.unh.edu:23571></a>
from the host
<br>
where you
run test
engine? <br>
<br>
If
behaviour the
same on the
next try and
you see that
test agent is
<br>
kept
running, could
you check
using <br>
<br>
# netstat
-tnlp <br>
<br>
that Test
Agent is
listening on
the port and
try to
establish TCP
<br>
connection
from test
agent using <br>
<br>
$ telnet <a
href="http://iol-dts-tester.dpdklab.iol.unh.edu" target="_blank"
moz-do-not-send="true">iol-dts-tester.dpdklab.iol.unh.edu</a>
<br>
<a
href="http://iol-dts-tester.dpdklab.iol.unh.edu:23571"
target="_blank" moz-do-not-send="true"><http://iol-dts-tester.dpdklab.iol.unh.edu:23571></a>
23571 <br>
<a
href="http://iol-dts-tester.dpdklab.iol.unh.edu:23571"
target="_blank" moz-do-not-send="true"><http://iol-dts-tester.dpdklab.iol.unh.edu:23571></a>
<br>
<br>
and check
if TCP
connection
could be
established. <br>
<br>
Another
idea is to
login Tester
under root as
testing does,
get <br>
start TA
command from
the log and
try it by
hands without
-n and <br>
remove
extra
escaping. <br>
<br>
# sudo
PATH=${PATH}:/tmp/linux_x86_root_76872_1692885663_1
<br>
LD_LIBRARY_PATH=${LD_LIBRARY_PATH}${LD_LIBRARY_PATH:+:}/tmp/linux_x86_root_76872_1692885663_1
/tmp/linux_x86_root_76872_1692885663_1/ta Peer 23571
host=iol-dts-tester.dpdklab.iol.unh.edu:port=23571:user=root:key=/opt/tsf/keys/id_ed25519:ssh_port=22:copy_timeout=15:kill_timeout=15:sudo=:shell=<br>
<br>
Hopefully
in this case
test agent
directory
remains in the
/tmp and <br>
you don't
need to copy
it as testing
does. <br>
May be
output could
shed some
light on
what's going
on. <br>
<br>
Andrew. <br>
<br>
On 8/24/23
17:30, Adam
Hassick wrote:
<br>
<blockquote
type="cite">
Hi Andrew, <br>
<br>
This is
the output
that I see in
the terminal
when this
failure <br>
occurs,
after the test
agent binaries
build and the
test engine <br>
starts: <br>
<br>
Platform
default build
- pass <br>
Simple RCF
consistency
check
succeeded <br>
--->>>
Starting
Logger...done
<br>
--->>>
Starting
RCF...rcf_net_engine_connect():
Connection
timed <br>
out <a
href="http://iol-dts-tester.dpdklab.iol.unh.edu:23571"
target="_blank" moz-do-not-send="true">iol-dts-tester.dpdklab.iol.unh.edu:23571</a>
<br>
<a
href="http://iol-dts-tester.dpdklab.iol.unh.edu:23571"
target="_blank" moz-do-not-send="true"><http://iol-dts-tester.dpdklab.iol.unh.edu:23571></a>
<br>
<br>
Then, it
hangs here
until I kill
the "te_rcf"
and "te_tee" <br>
processes.
I let it hang
for around 9
minutes. <br>
<br>
On the
tester host
(which appears
to be the Peer
agent), there
are <br>
four
processes that
I see running,
which look
like the test
agent <br>
processes.
<br>
<br>
ta.Peer is
an empty file.
I've attached
the log.txt
from this run.
<br>
<br>
- Adam <br>
<br>
On Thu,
Aug 24, 2023
at 4:22 AM
Andrew
Rybchenko <br>
<<a
href="mailto:andrew.rybchenko@oktetlabs.ru"
target="_blank" moz-do-not-send="true" class="moz-txt-link-freetext">andrew.rybchenko@oktetlabs.ru</a>
<br>
<a
href="mailto:andrew.rybchenko@oktetlabs.ru"
target="_blank" moz-do-not-send="true"><mailto:andrew.rybchenko@oktetlabs.ru></a>>
wrote: <br>
<br>
Hi
Adam, <br>
<br>
Yes,
TE_RCFUNIX_TIMEOUT
is in seconds.
I've
double-checked
<br>
that
it goes to
'copy_timeout'
in
ts-conf/rcf.conf.
<br>
Description in
in
doc/sphinx/pages/group_te_engine_rcf.rst
<br>
says
that
copy_timeout
is in seconds
and
implementation
in <br>
lib/rcfunix/rcfunix.c
passes the
value to
select()
tv_sec. <br>
Theoretically
select() could
be interrupted
by signal, but
I <br>
think
it is unlikely
here. <br>
<br>
I'm
not sure that
I understand
what do you
mean by RCF <br>
connection
timeout. Does
it happen on
TE startup
when RCF <br>
starts
test agents.
If so,
TE_RCFUNIX_TIMEOUT
could help. Or
<br>
does
it happen when
tests are in
progress, e.g.
in the middle
<br>
of a
test. If so,
TE_RCFUNIX_TIMEOUT
is unrelated
and most <br>
likely
either host
with test
agent dies or
test agent
itself <br>
crashes. It
would be
easier for me
if classify it
if you share <br>
text
log (log.txt,
full or just
corresponding
fragment with
<br>
some
context). Also
content of
ta.DPDK or
ta.Peer file <br>
depending on
which agent
has problems
could shed
some light. <br>
Corresponding
files contain
stdout/stderr
of test
agents. <br>
<br>
Andrew. <br>
<br>
On
8/23/23 17:45,
Adam Hassick
wrote: <br>
<blockquote
type="cite">
Hi Andrew, <br>
<br>
I've
set up a test
rig repository
here, and have
created <br>
configurations
for our
development
testbed based
off of the <br>
examples. <br>
We've
been able to
get the test
suite to run
manually on <br>
Mellanox CX5
devices once.
<br>
However, we
are running
into an issue
where, when
RCF starts, <br>
the
RCF connection
times out very
frequently. We
aren't sure <br>
why
this is the
case. <br>
It
works
sometimes, but
most of the
time when we
try to run <br>
the
test engine,
it encounters
this issue. <br>
I've
tried changing
the RCF port
by setting <br>
"TE_RCF_PORT=<some
port
number>"
and rebooting
the testbed <br>
machines.
Neither seems
to fix the
issue. <br>
<br>
It
also seems
like the
timeout takes
far longer
than 60 <br>
seconds, even
when running
"export
TE_RCFUNIX_TIMEOUT=60"
<br>
before
I try to run
the test
suite. <br>
I
assume the
unit for this
variable is
seconds? <br>
<br>
Thanks, <br>
Adam <br>
<br>
On
Mon, Aug 21,
2023 at
10:19 AM Adam
Hassick <br>
<<a
href="mailto:ahassick@iol.unh.edu" target="_blank"
moz-do-not-send="true"
class="moz-txt-link-freetext">ahassick@iol.unh.edu</a> <a
href="mailto:ahassick@iol.unh.edu"
target="_blank" moz-do-not-send="true"><mailto:ahassick@iol.unh.edu></a>>
wrote: <br>
<br>
Hi
Andrew, <br>
<br>
Thanks, I've
cloned the
example
repository and
will start <br>
setting up a
configuration
for our
development
testbed <br>
today. I'll
let you know
if I run into
any
difficulties <br>
or
have any
questions. <br>
<br>
-
Adam <br>
<br>
On
Sun, Aug 20,
2023 at
4:40 AM Andrew
Rybchenko <br>
<<a
href="mailto:andrew.rybchenko@oktetlabs.ru"
target="_blank" moz-do-not-send="true" class="moz-txt-link-freetext">andrew.rybchenko@oktetlabs.ru</a>
<br>
<a
href="mailto:andrew.rybchenko@oktetlabs.ru" target="_blank"
moz-do-not-send="true"><mailto:andrew.rybchenko@oktetlabs.ru></a>>
wrote: <br>
<br>
Hi Adam, <br>
<br>
I've published <br>
<a href="https://github.com/ts-factory/ts-rigs-sample"
target="_blank" moz-do-not-send="true" class="moz-txt-link-freetext">https://github.com/ts-factory/ts-rigs-sample</a>
<br>
<a href="https://github.com/ts-factory/ts-rigs-sample"
target="_blank" moz-do-not-send="true"><https://github.com/ts-factory/ts-rigs-sample></a>.
<br>
Hopefully it will help to define your test rigs and <br>
successfully run some tests manually. Feel free to <br>
ask any questions and I'll answer here and try to <br>
update documentation. <br>
<br>
Meanwhile I'll prepare missing bits for steps (2) and <br>
(3). <br>
Hopefully everything is in place for step (4), but we <br>
need to make steps (2) and (3) first. <br>
<br>
Andrew. <br>
<br>
On 8/18/23 21:40, Andrew Rybchenko wrote: <br>
<blockquote
type="cite">
Hi Adam, <br>
<br>
> I've conferred with the rest of the team, and we <br>
think it would be best to move forward with mainly <br>
option B. <br>
<br>
OK, I'll provide the sample on Monday for you. It is <br>
almost ready right now, but I need to double-check <br>
it before publishing. <br>
<br>
Regards, <br>
Andrew. <br>
<br>
On 8/17/23 20:03, Adam Hassick wrote: <br>
<blockquote
type="cite">
Hi Andrew, <br>
<br>
I'm adding the CI mailing list to this <br>
conversation. Others in the community might find <br>
this conversation valuable. <br>
<br>
We do want to run testing on a regular basis. The <br>
Jenkins integration will be very useful for us, as <br>
most of our CI is orchestrated by Jenkins. <br>
I've conferred with the rest of the team, and we <br>
think it would be best to move forward with mainly <br>
option B. <br>
If you would like to know anything about our <br>
testbeds that would help you with creating an <br>
example ts-rigs repo, I'd be happy to answer any <br>
questions you have. <br>
<br>
We have multiple test rigs (we call these <br>
"DUT-tester pairs") that we run our existing <br>
hardware testing on, with differing network <br>
hardware and CPU architecture. I figured this might <br>
be an important detail. <br>
<br>
Thanks, <br>
Adam <br>
<br>
On Thu, Aug 17, 2023 at 11:44 AM Andrew Rybchenko <br>
<<a href="mailto:andrew.rybchenko@oktetlabs.ru"
target="_blank"
moz-do-not-send="true" class="moz-txt-link-freetext">andrew.rybchenko@oktetlabs.ru</a>
<br>
<a href="mailto:andrew.rybchenko@oktetlabs.ru"
target="_blank"
moz-do-not-send="true"><mailto:andrew.rybchenko@oktetlabs.ru></a>>
wrote: <br>
<br>
Greatings Adam, <br>
<br>
I'm happy to hear that you're trying to bring <br>
it up. <br>
<br>
As I understand the final goal is to run it on <br>
regular basis. So, we need to make it properly <br>
from the very beginning. <br>
Bring up of all features consists of 4 steps: <br>
<br>
1. Create site-specific repository (we call it <br>
ts-rigs) which contains information about test <br>
rigs and other site-specific information like <br>
where to send mails, where to store logs etc. <br>
It is required for manual execution as well, <br>
since test rigs description is essential. I'll <br>
return to the topic below. <br>
<br>
2. Setup logs storage for automated runs. <br>
Basically it is a disk space plus apache2 web <br>
server with few CGI scripts which help a lot to <br>
save disk space. <br>
<br>
3. Setup Bublik web application which provides <br>
web interface to view testing results. Same as <br>
<a href="https://ts-factory.io/bublik"
target="_blank"
moz-do-not-send="true" class="moz-txt-link-freetext">https://ts-factory.io/bublik</a>
<br>
<a href="https://ts-factory.io/bublik"
target="_blank"
moz-do-not-send="true"><https://ts-factory.io/bublik></a> <br>
<br>
4. Setup Jenkins to run tests on regularly, <br>
save logs in log storage (2) and import it to <br>
bublik (3). <br>
<br>
Last few month we spent on our homework to make <br>
it simpler to bring up automated execution <br>
using Jenkins - <br>
<a href="https://github.com/ts-factory/te-jenkins"
target="_blank" moz-do-not-send="true" class="moz-txt-link-freetext">https://github.com/ts-factory/te-jenkins</a>
<br>
<a href="https://github.com/ts-factory/te-jenkins"
target="_blank" moz-do-not-send="true"><https://github.com/ts-factory/te-jenkins></a>
<br>
Corresponding bits in dpdk-ethdev-ts will be <br>
available tomorrow. <br>
<br>
Let's return to the step (1). <br>
<br>
Unfortunately there is no publicly available <br>
example of the ts-rigs repository since <br>
sensitive site-specific information is located <br>
there. But I'm ready to help you to create it <br>
for UNH. I see two options here: <br>
<br>
(A) I'll ask questions and based on your <br>
answers will create the first draft with my <br>
comments. <br>
<br>
(B) I'll make a template/example ts-rigs repo, <br>
publish it and you'll create UNH ts-rigs based <br>
on it. <br>
<br>
Of course, I'll help to debug and finally bring <br>
it up in any case. <br>
<br>
(A) is a bit simpler for me and you, but (B) is <br>
a bit more generic and will help other <br>
potential users to bring it up. <br>
We can combine (A)+(B). I.e. start from (A). <br>
What do you think? <br>
<br>
Thanks, <br>
Andrew. <br>
<br>
On 8/17/23 15:18, Konstantin Ushakov wrote: <br>
<blockquote
type="cite">
Greetings
Adam, <br>
<br>
<br>
Thanks for contacting us. I copy Andrew who <br>
would be happy to help <br>
<br>
Thanks, <br>
Konstantin <br>
<br>
<blockquote
type="cite">
On 16 Aug
2023, at
21:50, Adam
Hassick <br>
<a href="mailto:ahassick@iol.unh.edu"
target="_blank"
moz-do-not-send="true"><ahassick@iol.unh.edu></a> <br>
<a href="mailto:ahassick@iol.unh.edu"
target="_blank"
moz-do-not-send="true"><mailto:ahassick@iol.unh.edu></a> wrote: <br>
<br>
<br>
Greetings Konstantin, <br>
<br>
I am in the process of setting up the DPDK <br>
Poll Mode Driver test suite as an addition to <br>
our testing coverage for DPDK at the UNH lab. <br>
<br>
I have some questions about how to set the <br>
test suite arguments. <br>
<br>
I have been able to configure the Test Engine <br>
to connect to the hosts in the testbed. The <br>
RCF, Configurator, and Tester all begin to <br>
run, however the prelude of the test suite <br>
fails to run. <br>
<br>
<a
href="https://ts-factory.io/doc/dpdk-ethdev-ts/index.html#test-parameters"
target="_blank" moz-do-not-send="true" class="moz-txt-link-freetext">https://ts-factory.io/doc/dpdk-ethdev-ts/index.html#test-parameters</a>
<a
href="https://ts-factory.io/doc/dpdk-ethdev-ts/index.html#test-parameters"
target="_blank" moz-do-not-send="true"><https://ts-factory.io/doc/dpdk-ethdev-ts/index.html#test-parameters></a>
<br>
<br>
The documentation mentions that there are <br>
several test parameters for the test suite, <br>
like for the IUT test link MAC, etc. These <br>
seem like they would need to be set somewhere <br>
to run many of the tests. <br>
<br>
I see in the Test Engine documentation, there <br>
are instructions on how to create new <br>
parameters for test suites in the Tester <br>
configuration, but there is nothing in the <br>
user guide or in the Tester guide for how to <br>
set the arguments for the parameters when <br>
running the test suite that I can find. I'm <br>
not sure if I need to write my own Tester <br>
config, or if I should be setting these in <br>
some other way. <br>
<br>
How should these values be set? <br>
<br>
I'm also not sure what environment <br>
variables/arguments are strictly necessary or <br>
which are optional. <br>
<br>
Regards, <br>
Adam <br>
<br>
-- *Adam Hassick* <br>
Senior Developer <br>
UNH InterOperability Lab <br>
<a href="mailto:ahassick@iol.unh.edu"
target="_blank"
moz-do-not-send="true" class="moz-txt-link-freetext">ahassick@iol.unh.edu</a>
<br>
<a href="mailto:ahassick@iol.unh.edu"
target="_blank"
moz-do-not-send="true"><mailto:ahassick@iol.unh.edu></a> <br>
<a href="http://iol.unh.edu" target="_blank"
moz-do-not-send="true">iol.unh.edu</a>
<a
href="https://www.iol.unh.edu/"
target="_blank" moz-do-not-send="true"><https://www.iol.unh.edu/></a>
<br>
+1 (603) 475-8248 <br>
</blockquote>
</blockquote>
<br>
<br>
<br>
-- *Adam Hassick* <br>
Senior Developer <br>
UNH InterOperability Lab <br>
<a href="mailto:ahassick@iol.unh.edu" target="_blank"
moz-do-not-send="true"
class="moz-txt-link-freetext">ahassick@iol.unh.edu</a> <a
href="mailto:ahassick@iol.unh.edu"
target="_blank" moz-do-not-send="true"><mailto:ahassick@iol.unh.edu></a>
<br>
<a href="http://iol.unh.edu" target="_blank"
moz-do-not-send="true">iol.unh.edu</a>
<a
href="https://www.iol.unh.edu/"
target="_blank" moz-do-not-send="true"><https://www.iol.unh.edu/></a>
<br>
+1 (603) 475-8248 <br>
</blockquote>
<br>
</blockquote>
<br>
<br>
<br>
--
*Adam Hassick*
<br>
Senior
Developer <br>
UNH
InterOperability
Lab <br>
<a
href="mailto:ahassick@iol.unh.edu" target="_blank"
moz-do-not-send="true"
class="moz-txt-link-freetext">ahassick@iol.unh.edu</a> <a
href="mailto:ahassick@iol.unh.edu"
target="_blank" moz-do-not-send="true"><mailto:ahassick@iol.unh.edu></a>
<br>
<a
href="http://iol.unh.edu" target="_blank" moz-do-not-send="true">iol.unh.edu</a>
<a
href="https://www.iol.unh.edu/"
target="_blank" moz-do-not-send="true"><https://www.iol.unh.edu/></a>
<br>
+1
(603) 475-8248
<br>
<br>
<br>
<br>
--
*Adam
Hassick* <br>
Senior
Developer <br>
UNH
InterOperability
Lab <br>
<a
href="mailto:ahassick@iol.unh.edu"
target="_blank" moz-do-not-send="true" class="moz-txt-link-freetext">ahassick@iol.unh.edu</a>
<a
href="mailto:ahassick@iol.unh.edu"
target="_blank" moz-do-not-send="true"><mailto:ahassick@iol.unh.edu></a>
<br>
<a
href="http://iol.unh.edu"
target="_blank" moz-do-not-send="true">iol.unh.edu</a> <a
href="https://www.iol.unh.edu/"
target="_blank" moz-do-not-send="true"><https://www.iol.unh.edu/></a>
<br>
+1
(603) 475-8248
<br>
</blockquote>
<br>
<br>
<br>
--
*Adam Hassick*
<br>
Senior
Developer <br>
UNH
InterOperability
Lab <br>
<a
href="mailto:ahassick@iol.unh.edu"
target="_blank" moz-do-not-send="true" class="moz-txt-link-freetext">ahassick@iol.unh.edu</a>
<a
href="mailto:ahassick@iol.unh.edu"
target="_blank" moz-do-not-send="true"><mailto:ahassick@iol.unh.edu></a>
<br>
<a
href="http://iol.unh.edu"
target="_blank" moz-do-not-send="true">iol.unh.edu</a> <a
href="https://www.iol.unh.edu/"
target="_blank" moz-do-not-send="true"><https://www.iol.unh.edu/></a>
<br>
+1 (603)
475-8248 <br>
</blockquote>
<br>
<br>
<br>
-- <br>
*Adam Hassick*
<br>
Senior
Developer <br>
UNH
InterOperability
Lab <br>
<a
href="mailto:ahassick@iol.unh.edu"
target="_blank" moz-do-not-send="true" class="moz-txt-link-freetext">ahassick@iol.unh.edu</a>
<a
href="mailto:ahassick@iol.unh.edu"
target="_blank" moz-do-not-send="true"><mailto:ahassick@iol.unh.edu></a>
<br>
<a
href="http://iol.unh.edu"
target="_blank" moz-do-not-send="true">iol.unh.edu</a> <a
href="https://www.iol.unh.edu/"
target="_blank" moz-do-not-send="true"><https://www.iol.unh.edu/></a>
<br>
+1 (603)
475-8248 <br>
</blockquote>
<br>
</blockquote>
<br>
</div>
</blockquote>
</div>
<br
clear="all">
<br>
<span
class="gmail_signature_prefix">--
</span><br>
<div dir="ltr"
class="gmail_signature">
<div dir="ltr">
<div>
<div><b><span
style="background-color:rgb(255,255,255)"><span
style="color:rgb(102,102,102)">Adam
Hassick</span></span></b><br>
</div>
<span
style="color:rgb(102,102,102)"></span></div>
<div><span
style="color:rgb(102,102,102)">Senior
Developer</span></div>
<div><span
style="color:rgb(102,102,102)"><span
style="color:rgb(11,83,148)"><span
style="background-color:rgb(255,255,255)">UNH
InterOperability Lab</span></span></span><span
style="color:rgb(102,102,102)"></span></div>
<div><span
style="color:rgb(102,102,102)"><a
href="mailto:ahassick@iol.unh.edu" target="_blank"
moz-do-not-send="true"
class="moz-txt-link-freetext">ahassick@iol.unh.edu</a><br>
</span></div>
<div><span
style="color:rgb(102,102,102)"><a
href="https://www.iol.unh.edu/" target="_blank" moz-do-not-send="true">iol.unh.edu</a><br>
</span></div>
+1 (603)
475-8248<br>
</div>
</div>
</blockquote>
<br>
</blockquote>
<br>
</div>
</blockquote>
</div>
<br clear="all">
<br>
<span
class="gmail_signature_prefix">--
</span><br>
<div dir="ltr"
class="gmail_signature">
<div dir="ltr">
<div>
<div><b><span
style="background-color:rgb(255,255,255)"><span
style="color:rgb(102,102,102)">Adam Hassick</span></span></b><br>
</div>
<span
style="color:rgb(102,102,102)"></span></div>
<div><span
style="color:rgb(102,102,102)">Senior
Developer</span></div>
<div><span
style="color:rgb(102,102,102)"><span
style="color:rgb(11,83,148)"><span
style="background-color:rgb(255,255,255)">UNH
InterOperability Lab</span></span></span><span
style="color:rgb(102,102,102)"></span></div>
<div><span
style="color:rgb(102,102,102)"><a
href="mailto:ahassick@iol.unh.edu" target="_blank"
moz-do-not-send="true"
class="moz-txt-link-freetext">ahassick@iol.unh.edu</a><br>
</span></div>
<div><span
style="color:rgb(102,102,102)"><a
href="https://www.iol.unh.edu/" target="_blank" moz-do-not-send="true">iol.unh.edu</a><br>
</span></div>
+1 (603) 475-8248<br>
</div>
</div>
</blockquote>
<br>
</blockquote>
<br>
</div>
</blockquote>
</div>
<br clear="all">
<br>
<span
class="gmail_signature_prefix">--
</span><br>
<div dir="ltr"
class="gmail_signature">
<div dir="ltr">
<div>
<div><b><span
style="background-color:rgb(255,255,255)"><span
style="color:rgb(102,102,102)">Adam Hassick</span></span></b><br>
</div>
<span
style="color:rgb(102,102,102)"></span></div>
<div><span
style="color:rgb(102,102,102)">Senior
Developer</span></div>
<div><span
style="color:rgb(102,102,102)"><span
style="color:rgb(11,83,148)"><span
style="background-color:rgb(255,255,255)">UNH
InterOperability Lab</span></span></span><span
style="color:rgb(102,102,102)"></span></div>
<div><span
style="color:rgb(102,102,102)"><a
href="mailto:ahassick@iol.unh.edu" target="_blank"
moz-do-not-send="true"
class="moz-txt-link-freetext">ahassick@iol.unh.edu</a><br>
</span></div>
<div><span
style="color:rgb(102,102,102)"><a
href="https://www.iol.unh.edu/" target="_blank" moz-do-not-send="true">iol.unh.edu</a><br>
</span></div>
+1 (603) 475-8248<br>
</div>
</div>
</blockquote>
<br>
</div>
</blockquote>
</div>
</div>
</div>
</blockquote>
<br>
</div>
</blockquote>
</div>
<br clear="all">
<br>
<span class="gmail_signature_prefix">-- </span><br>
<div dir="ltr" class="gmail_signature">
<div dir="ltr">
<div>
<div><b><span
style="background-color:rgb(255,255,255)"><span
style="color:rgb(102,102,102)">Adam
Hassick</span></span></b><br>
</div>
<span style="color:rgb(102,102,102)"></span></div>
<div><span style="color:rgb(102,102,102)">Senior
Developer</span></div>
<div><span style="color:rgb(102,102,102)"><span
style="color:rgb(11,83,148)"><span
style="background-color:rgb(255,255,255)">UNH
InterOperability Lab</span></span></span><span
style="color:rgb(102,102,102)"></span></div>
<div><span style="color:rgb(102,102,102)"><a
href="mailto:ahassick@iol.unh.edu"
target="_blank" moz-do-not-send="true"
class="moz-txt-link-freetext">ahassick@iol.unh.edu</a><br>
</span></div>
<div><span style="color:rgb(102,102,102)"><a
href="https://www.iol.unh.edu/"
target="_blank" moz-do-not-send="true">iol.unh.edu</a><br>
</span></div>
+1 (603) 475-8248<br>
</div>
</div>
</blockquote>
<br>
</div>
</blockquote>
</div>
<br clear="all">
<br>
<span class="gmail_signature_prefix">-- </span><br>
<div dir="ltr" class="gmail_signature">
<div dir="ltr">
<div>
<div><b><span
style="background-color:rgb(255,255,255)"><span
style="color:rgb(102,102,102)">Adam Hassick</span></span></b><br>
</div>
<span style="color:rgb(102,102,102)"></span></div>
<div><span style="color:rgb(102,102,102)">Senior
Developer</span></div>
<div><span style="color:rgb(102,102,102)"><span
style="color:rgb(11,83,148)"><span
style="background-color:rgb(255,255,255)">UNH
InterOperability Lab</span></span></span><span
style="color:rgb(102,102,102)"></span></div>
<div><span style="color:rgb(102,102,102)"><a
href="mailto:ahassick@iol.unh.edu" target="_blank"
moz-do-not-send="true"
class="moz-txt-link-freetext">ahassick@iol.unh.edu</a><br>
</span></div>
<div><span style="color:rgb(102,102,102)"><a
href="https://www.iol.unh.edu/" target="_blank"
moz-do-not-send="true">iol.unh.edu</a><br>
</span></div>
+1 (603) 475-8248<br>
</div>
</div>
</blockquote>
</div>
<br clear="all">
<br>
<span class="gmail_signature_prefix">-- </span><br>
<div dir="ltr" class="gmail_signature">
<div dir="ltr">
<div>
<div><b><span style="background-color:rgb(255,255,255)"><span
style="color:rgb(102,102,102)">Adam Hassick</span></span></b><br>
</div>
<span style="color:rgb(102,102,102)"></span></div>
<div><span style="color:rgb(102,102,102)">Senior Developer</span></div>
<div><span style="color:rgb(102,102,102)"><span
style="color:rgb(11,83,148)"><span
style="background-color:rgb(255,255,255)">UNH
InterOperability Lab</span></span></span><span
style="color:rgb(102,102,102)"></span></div>
<div><span style="color:rgb(102,102,102)"><a
href="mailto:ahassick@iol.unh.edu" target="_blank"
moz-do-not-send="true" class="moz-txt-link-freetext">ahassick@iol.unh.edu</a><br>
</span></div>
<div><span style="color:rgb(102,102,102)"><a
href="https://www.iol.unh.edu/" target="_blank"
moz-do-not-send="true">iol.unh.edu</a><br>
</span></div>
+1 (603) 475-8248<br>
</div>
</div>
</blockquote>
<br>
</blockquote>
<br>
</body>
</html>