<div dir="ltr"><div dir="ltr"><br></div><br><div class="gmail_quote gmail_quote_container"><div dir="ltr" class="gmail_attr">On Thu, Jun 12, 2025 at 4:12 PM Dean Marx <<a href="mailto:dmarx@iol.unh.edu">dmarx@iol.unh.edu</a>> wrote:<br></div><blockquote class="gmail_quote" style="margin:0px 0px 0px 0.8ex;border-left:1px solid rgb(204,204,204);padding-left:1ex">Rearrange the topology and DPDK setup/teardown calls during<br>
test runs to ensure the devbind script is not called<br>
while the DPDK tmp directory doesn't exist.<br>
<br>
Fixes: 4cef16f1f0a4 ("dts: improve port handling")<br>
<br>
Signed-off-by: Dean Marx <<a href="mailto:dmarx@iol.unh.edu" target="_blank">dmarx@iol.unh.edu</a>><br>
---<br>
dts/framework/test_run.py | 4 ++--<br>
1 file changed, 2 insertions(+), 2 deletions(-)<br>
<br>
diff --git a/dts/framework/test_run.py b/dts/framework/test_run.py<br>
index cff0085317..4287028b4b 100644<br>
--- a/dts/framework/test_run.py<br>
+++ b/dts/framework/test_run.py<br>
@@ -344,8 +344,8 @@ def next(self) -> State | None:<br>
<br>
test_run.ctx.sut_node.setup()<br>
test_run.ctx.tg_node.setup()<br>
- test_run.ctx.topology.setup()<br>
test_run.ctx.dpdk.setup()<br>
+ test_run.ctx.topology.setup()<br>
test_run.ctx.tg.setup(test_run.ctx.topology)<br>
<br>
self.result.ports = test_run.ctx.topology.sut_ports + test_run.ctx.topology.tg_ports<br>
@@ -433,8 +433,8 @@ def next(self) -> State | None:<br>
"""Next state."""<br>
self.test_run.ctx.shell_pool.terminate_current_pool()<br>
self.test_run.ctx.tg.teardown()<br>
- self.test_run.ctx.dpdk.teardown()<br>
self.test_run.ctx.topology.teardown()<br>
+ self.test_run.ctx.dpdk.teardown()<br>
self.test_run.ctx.tg_node.teardown()<br>
self.test_run.ctx.sut_node.teardown()<br>
self.result.update_teardown(Result.PASS)<br>
-- <br>
2.49.0<br>
<br></blockquote><div><br></div><div>Thanks this makes sense in principle but I think there is an additional issue, which I saw when I just tested your patch on 2 XL710 NICs connected both on the Braum server.</div><div><br></div><div>Here is the execution order from the testrun</div><div><br></div><div>1. testrun init creates DPDKRuntimeEnvironment with _node=sutnote like: </div><div><br></div><div>dpdk_runtime_env = DPDKRuntimeEnvironment(config.dpdk, sut_node, dpdk_build_env)</div><div><br></div><div>which is then added to ctx.</div><div><br></div><div>2. TestRunSetup calls test_run.ctx.dpdk.setup() which calls _prepare_devbind_script() which adds the devbind_script_path to _node.main_session, but _node in this case is the sut node. I don't believe there is the code path for adding devbind_script_path to the tg node main_session, so later at topology.setup() we call self._setup_ports("sut") which runs fine, and then error on self._setup_ports("tg") because again the tg node main session is lacking the devbind_script_path attribute. I think you will need to address this second part for your patch to be complete. In any case, let's chat in the morning and figure out what a v2 of this bugfix can be.</div><div><br></div><div>See this stacktrace:</div><div><br>2025/06/16 02:59:03 - test_run_setup - dts.braum-intel-40g-SUT - INFO - Sending: 'lshw -quiet -json -C network'<br>2025/06/16 02:59:04 - test_run_setup - dts.braum-intel-40g-TG - INFO - Sending: 'lshw -quiet -json -C network'<br>2025/06/16 02:59:05 - test_run_setup - dts - ERROR - A CRITICAL ERROR has occurred during test run setup. Unrecoverable state reached, shutting down.<br>2025/06/16 02:59:05 - test_run_setup - dts - ERROR - An unexpected error has occurred.<br>Traceback (most recent call last):<br> File "/dpdk/dts/framework/runner.py", line 81, in run<br> test_run.spin()<br> File "/dpdk/dts/framework/test_run.py", line 242, in spin<br> next_state = self.state.handle_exception(e)<br> ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^<br> File "/dpdk/dts/framework/test_run.py", line 240, in spin<br> next_state = self.state.next()<br> ^^^^^^^^^^^^^^^^^<br> File "/dpdk/dts/framework/test_run.py", line 348, in next<br> test_run.ctx.topology.setup()<br> File "/dpdk/dts/framework/testbed_model/topology.py", line 130, in setup<br> self._setup_ports("tg")<br> File "/dpdk/dts/framework/testbed_model/topology.py", line 154, in _setup_ports<br> self._bind_ports_to_drivers(node, ports, "kernel")<br> File "/dpdk/dts/framework/testbed_model/topology.py", line 205, in _bind_ports_to_drivers<br> node.main_session.bind_ports_to_driver(ports, driver_name)<br> File "/dpdk/dts/framework/testbed_model/linux_session.py", line 187, in bind_ports_to_driver<br> f"{self.devbind_script_path} -b {driver_name} --force {ports_pci_addrs}",<br> ^^^^^^^^^^^^^^^^^^^^^^^^<br> File "/usr/lib/python3.12/functools.py", line 995, in __get__<br> val = self.func(instance)<br> ^^^^^^^^^^^^^^^^^^^<br> File "/dpdk/dts/framework/testbed_model/linux_session.py", line 212, in devbind_script_path<br> raise InternalError("Accessed devbind script path before setup.")<br>framework.exception.InternalError: Accessed devbind script path before setup.<br>2025/06/16 02:59:05 - test_run_setup - dts - INFO - Moving from stage 'test_run_setup' to stage 'post_run'.<br>2025/06/16 02:59:05 - post_run - dts - INFO - DTS execution has ended.</div><div><br></div><div> </div></div></div>