[PATCH v4 00/11] dts: add test skipping based on capabilities
Juraj Linkeš
juraj.linkes at pantheon.tech
Mon Sep 30 15:43:17 CEST 2024
On 24. 9. 2024 10:20, Juraj Linkeš wrote:
> Add an automated way to gather available capabilities of the tested
> hardware and skip test suites or cases which require capabilities that
> are not available.
>
> This is done through two decorators:
> 1. The first marks a test suite method as test case. This populates the
> default attributes of each test case.
> 2. The seconds adds the required capabilities to a test suite or case,
> using the attributes from 1).
>
> Two types of capabilities are added:
> 1. NIC capabilities. These are gathered once DPDK has been built because
> we use testpmd for this. It's possible to add a function that will
> add configuration before assessing capabilities associated with the
> function. This is because some capabilities return different status
> with different configuration present.
> 2. The topology capability. Each test case is marked as requiring a
> default topology. The required topology of a test case (or a whole
> test suite) may be change with the second decorator.
>
> This is how it all works:
> 1. The required capabilities are first all gathered from all test suites
> and test cases.
> 2. The list of required capabilities is divided into supported and
> unsupported capabilities. In this step, the probing of hardware
> and/or anything else that needs to happen to gauge whether a
> capability is supported is done.
> 3. Each test suite and test case is then marked to be skipped if any of
> their required capabilities are not supported.
>
> v4:
> Rebased on next-dts.
> Adjusted the decorator for setting and reverting 9000 MTU to be
> parametrized.
> Added references to DPDK libraries and testpmd code to testpmd command
> parsing methods.
> Updated capability docstrings. Moved the capability decorator to
> NicCapability.
>
> Juraj Linkeš (11):
> dts: add the aenum dependency
> dts: add test case decorators
> dts: add mechanism to skip test cases or suites
> dts: add support for simpler topologies
> dts: add basic capability support
> dts: add NIC capability support
> dts: add NIC capabilities from show rxq info
> dts: add topology capability
> doc: add DTS capability doc sources
> dts: add Rx offload capabilities
> dts: add NIC capabilities from show port info
>
> .../framework.testbed_model.capability.rst | 6 +
> doc/api/dts/framework.testbed_model.rst | 2 +
> .../dts/framework.testbed_model.topology.rst | 6 +
> dts/framework/remote_session/testpmd_shell.py | 461 +++++++++++++++-
> dts/framework/runner.py | 155 +++---
> dts/framework/test_result.py | 119 ++--
> dts/framework/test_suite.py | 161 +++++-
> dts/framework/testbed_model/capability.py | 507 ++++++++++++++++++
> dts/framework/testbed_model/node.py | 2 +-
> dts/framework/testbed_model/port.py | 4 +-
> dts/framework/testbed_model/topology.py | 127 +++++
> dts/poetry.lock | 14 +-
> dts/pyproject.toml | 1 +
> dts/tests/TestSuite_hello_world.py | 10 +-
> dts/tests/TestSuite_os_udp.py | 3 +-
> dts/tests/TestSuite_pmd_buffer_scatter.py | 14 +-
> dts/tests/TestSuite_smoke_tests.py | 8 +-
> 17 files changed, 1447 insertions(+), 153 deletions(-)
> create mode 100644 doc/api/dts/framework.testbed_model.capability.rst
> create mode 100644 doc/api/dts/framework.testbed_model.topology.rst
> create mode 100644 dts/framework/testbed_model/capability.py
> create mode 100644 dts/framework/testbed_model/topology.py
>
Applied to next-dts, thanks for the reviews.
More information about the dev
mailing list