[PATCH v3 1/2] dts: add code coverage reporting to DTS
Koushik Bhargav Nimoji
knimoji at iol.unh.edu
Thu Jun 25 19:41:36 CEST 2026
On Wed, Jun 24, 2026 at 10:34 PM Patrick Robb <patrickrobb1997 at gmail.com>
wrote:
>
>
> On Mon, Jun 22, 2026 at 1:02 PM Koushik Bhargav Nimoji <
> knimoji at iol.unh.edu> wrote:
>
>> Previously, DTS had no code coverage. This patch adds a command line
>> argument in order to build DPDK with code coverage enabled. This allows
>> users to create and view code coverage reports of what code and functions
>> were called during a DTS run.
>>
>> Signed-off-by: Koushik Bhargav Nimoji <knimoji at iol.unh.edu>
>> ---
>> v2:
>> *Fixed error in lcov/gcov tool detection
>> v3:
>> *Fixed type hints and error message typos
>> ---
>> .mailmap | 1 +
>> doc/guides/tools/dts.rst | 15 +++++++++++++
>> dts/README.md | 5 +++++
>> dts/framework/remote_session/dpdk.py | 19 ++++++++++++++++
>> .../remote_session/remote_session.py | 5 ++++-
>> dts/framework/settings.py | 10 +++++++++
>> dts/framework/testbed_model/os_session.py | 10 +++++++++
>> dts/framework/testbed_model/posix_session.py | 22 +++++++++++++++++++
>> dts/framework/utils.py | 8 +++++++
>> 9 files changed, 94 insertions(+), 1 deletion(-)
>>
>> diff --git a/.mailmap b/.mailmap
>> index e052b85213..a1209150ad 100644
>> --- a/.mailmap
>> +++ b/.mailmap
>> @@ -877,6 +877,7 @@ Klaus Degner <kd at allegro-packets.com>
>> Kommula Shiva Shankar <kshankar at marvell.com>
>> Konstantin Ananyev <konstantin.ananyev at huawei.com> <
>> konstantin.v.ananyev at yandex.ru>
>> Konstantin Ananyev <konstantin.ananyev at huawei.com> <
>> konstantin.ananyev at intel.com>
>> +Koushik Bhargav Nimoji <knimoji at iol.unh.edu>
>> Krishna Murthy <krishna.j.murthy at intel.com>
>> Krzysztof Galazka <krzysztof.galazka at intel.com>
>> Krzysztof Kanas <kkanas at marvell.com> <krzysztof.kanas at caviumnetworks.com
>> >
>> diff --git a/doc/guides/tools/dts.rst b/doc/guides/tools/dts.rst
>> index 5b9a348016..a838a317ee 100644
>> --- a/doc/guides/tools/dts.rst
>> +++ b/doc/guides/tools/dts.rst
>> @@ -352,6 +352,10 @@ DTS is run with ``main.py`` located in the ``dts``
>> directory using the ``poetry
>> --precompiled-build-dir DIR_NAME
>> [DTS_PRECOMPILED_BUILD_DIR] Define the
>> subdirectory under the DPDK tree root directory or tarball where the pre-
>> compiled binaries are located. (default: None)
>> + --code-coverage Builds DPDK on the SUT node with code
>> coverage enabled. Generates a code coverage report which can be found on
>> + the local filesystem at
>> dts/output/coverage_reports/meson-logs/coveragereport/index.html, or the
>> specified output
>>
>
> at the DTS execution host's local filesystem
>
> I realize you are presumably concating what gcov/lcov gives you but can
> the dts/output/coverage_reports/meson-logs/coveragereport/index.html,
> path be shortened? Seems like 2-3 of those middle dir names can be dropped
> hah. Not a big deal if left as is for any reason.
>
I looked into shortening the path initially, but based on the structure of
the report and its components I believe it would be most simple to keep it
as is.
>
>> + directory. To use code coverage, please
>> ensure lcov v1.15 and gcov v8.0 or higher (included in gcc package) are
>> + installed on the SUT node.
>>
>>
>> The brackets contain the names of environment variables that set the
>> same thing.
>> @@ -367,6 +371,17 @@ Results are stored in the output dir by default
>> which be changed with the ``--output-dir`` command line argument.
>> The results contain basic statistics of passed/failed test cases and
>> DPDK version.
>>
>> +Code Coverage
>> +~~~~~~~~~~~~~
>> +
>> +DTS has the ablilty to track code usage during test runs, and generate
>> an HTML
>>
>
> I'm sure it's obvious to most readers what coverage we are talking about
> here, but why not just explicitly say DTS can generate coverage reports
> which show the code coverage % for DPDK libraries and drivers touched
> during the testsuite(s) execution? It never hurts to be extra clear. :)
>
> VERY IMPORTANT: You need to explain what the code coverage behavior is. Is
> it a code coverage report per testrun? or per testsuite?
>
> If it is per testrun, what happens if we use a prebuilt DPDK dir? Then do
> coverage stats bleed over between runs because the build dir is preserved?
> (happy to talk about this tomorrow if I'm not phrasing it clearly).
>
> +coverage report with that data. This can be done by using the
>> "--code-coverage"
>> +CLI parameter when running DTS.
>> +
>> +To use code coverage, please make sure the following dependencies are
>> available
>> +on the SUT node:
>> +- lcov v1.15
>>
>
> code says 1.15 or greater
>
>
>> +- gcov v8.0 or greater (included in gcc package)
>>
>> Contributing to DTS
>> -------------------
>> diff --git a/dts/README.md b/dts/README.md
>> index d257b7a167..51f824e077 100644
>> --- a/dts/README.md
>> +++ b/dts/README.md
>> @@ -64,6 +64,11 @@ $ poetry run ./main.py
>> These commands will give you a bash shell inside a docker container
>> with all DTS Python dependencies installed.
>>
>> +# Code Coverage
>> +
>> +To generate code coverage reports, ensure the SUT has lcov v1.15 and
>> gcov v8.0 or greater
>> +installed, and that DTS is run using the '--code-coverage' argument.
>>
>
> Not that I'm opposed, but it is interesting to me that we are exposing
> this toggle as a flag but not as a test_run.yaml option. I was about to
> suggest adding a test_run.yaml boolean field for it but... maybe we need to
> relax on the amount of fields we put in there. It might be better for some
> of the more "infrequently used" options to be flag only, for
> readability reasons. Happy to defer to your judgement here.
>
I agree, as code coverage is more so an "add-on" to a DTS run. The
components of the test_run.yaml are more so required for the test run, so
it would be better to keep it as a flag.
> +
>> ## Visual Studio Code
>>
>> Usage of VScode devcontainers is NOT required for developing on DTS and
>> running DTS,
>> diff --git a/dts/framework/remote_session/dpdk.py
>> b/dts/framework/remote_session/dpdk.py
>> index c3575cfcaf..865f97f6ca 100644
>> --- a/dts/framework/remote_session/dpdk.py
>> +++ b/dts/framework/remote_session/dpdk.py
>> @@ -29,6 +29,7 @@
>> from framework.logger import DTSLogger, get_dts_logger
>> from framework.params.eal import EalParams
>> from framework.remote_session.remote_session import CommandResult
>> +from framework.settings import SETTINGS
>> from framework.testbed_model.cpu import LogicalCore, LogicalCoreCount,
>> LogicalCoreList, lcore_filter
>> from framework.testbed_model.node import Node
>> from framework.testbed_model.os_session import OSSession
>> @@ -107,7 +108,22 @@ def teardown(self) -> None:
>> """Teardown the DPDK build on the target node.
>>
>> Removes the DPDK tree and/or build directory/tarball depending
>> on the configuration.
>> + If code coverage is enabled, the coverage report and .info file
>> are generated and
>> + copied onto the local filesystem before teardown.
>> """
>> + if SETTINGS.code_coverage:
>> + report_folder = PurePath(self.remote_dpdk_build_dir /
>> "meson-logs")
>> + output_dir = SETTINGS.output_dir
>> + Path(output_dir).mkdir(parents=True, exist_ok=True)
>> +
>> + coverage_status =
>> self._session.generate_coverage_report(self.remote_dpdk_build_dir)
>> + if coverage_status:
>> + self._session.copy_dir_from(report_folder, output_dir)
>> + self._logger.info(
>> + "Coverage HTML report generated, "
>> + f"available at
>> {output_dir}/meson-logs/coveragereports/index.html"
>> + )
>> +
>> match self.config.dpdk_location:
>> case LocalDPDKTreeLocation():
>>
>> self._node.main_session.remove_remote_dir(self.remote_dpdk_tree_path)
>> @@ -272,6 +288,9 @@ def _build_dpdk(self) -> None:
>> else:
>> meson_args = MesonArgs(default_library="static",
>> libdir="lib")
>>
>> + if SETTINGS.code_coverage:
>> + meson_args._add_arg("-Db_coverage=true")
>> +
>> self._session.build_dpdk(
>> self._env_vars,
>> meson_args,
>> diff --git a/dts/framework/remote_session/remote_session.py
>> b/dts/framework/remote_session/remote_session.py
>> index 158325bb7f..d2440dc2d8 100644
>> --- a/dts/framework/remote_session/remote_session.py
>> +++ b/dts/framework/remote_session/remote_session.py
>> @@ -252,7 +252,10 @@ def copy_from(self, source_file: str | PurePath,
>> destination_dir: str | Path) ->
>> destination_dir: The directory path on the local filesystem
>> where the `source_file`
>> will be saved.
>> """
>> - self.session.get(str(source_file), str(destination_dir))
>> + source_file = PurePath(source_file)
>> + destination_dir = Path(destination_dir)
>> + local_path = destination_dir / source_file.name
>> + self.session.get(str(source_file), str(local_path))
>>
>> def copy_to(self, source_file: str | Path, destination_dir: str |
>> PurePath) -> None:
>> """Copy a file from local filesystem to the remote Node.
>> diff --git a/dts/framework/settings.py b/dts/framework/settings.py
>> index b08373b7ea..7df535bd84 100644
>> --- a/dts/framework/settings.py
>> +++ b/dts/framework/settings.py
>> @@ -159,6 +159,8 @@ class Settings:
>> re_run: int = 0
>> #:
>> random_seed: int | None = None
>> + #:
>> + code_coverage: bool = False
>>
>>
>> SETTINGS: Settings = Settings()
>> @@ -489,6 +491,14 @@ def _get_parser() -> _DTSArgumentParser:
>> )
>> _add_env_var_to_action(action)
>>
>> + action = parser.add_argument(
>> + "--code-coverage",
>> + action="store_true",
>> + default=False,
>> + help="Used to build DPDK with code coverage enabled.",
>> + )
>> + _add_env_var_to_action(action)
>> +
>> return parser
>>
>>
>> diff --git a/dts/framework/testbed_model/os_session.py
>> b/dts/framework/testbed_model/os_session.py
>> index 2c267afed1..742b074948 100644
>> --- a/dts/framework/testbed_model/os_session.py
>> +++ b/dts/framework/testbed_model/os_session.py
>> @@ -480,6 +480,16 @@ def build_dpdk(
>> timeout: Wait at most this long in seconds for the build
>> execution to complete.
>> """
>>
>> + @abstractmethod
>> + def generate_coverage_report(self, remote_build_dir: PurePath |
>> None) -> bool:
>> + """Generates a code coverage report for a DTS run.
>> +
>> + Args:
>> + remote_build_dir: The remote DPDK build directory
>> + Returns:
>> + Whether the coverage report was able to be created or not.
>> + """
>> +
>> @abstractmethod
>> def get_dpdk_version(self, version_path: str | PurePath) -> str:
>> """Inspect the DPDK version on the remote node.
>> diff --git a/dts/framework/testbed_model/posix_session.py
>> b/dts/framework/testbed_model/posix_session.py
>> index dec952685a..d18ce27de2 100644
>> --- a/dts/framework/testbed_model/posix_session.py
>> +++ b/dts/framework/testbed_model/posix_session.py
>> @@ -295,6 +295,28 @@ def build_dpdk(
>> except RemoteCommandExecutionError as e:
>> raise DPDKBuildError(f"DPDK build failed when doing
>> '{e.command}'.")
>>
>> + def generate_coverage_report(self, remote_build_dir: PurePath |
>> None) -> bool:
>> + """Overrides
>> :meth:`~.os_session.OSSession.generate_coverage_report`."""
>> + command_result = self.send_command(r"lcov --version | grep -oP
>> '\d+\.\d+'")
>> + lcov_version = float(
>> + command_result.stdout if command_result.return_code == 0 and
>> command_result else -1
>> + )
>> + command_result = self.send_command(
>> + r"gcov --version | head -n 1 | grep -oP '\d+\.\d+' | tail -n
>> 1"
>> + )
>> + gcov_version = float(
>> + command_result.stdout if command_result.return_code == 0 and
>> command_result else -1
>> + )
>> +
>> + if lcov_version >= 1.15 and gcov_version >= 8.0:
>> + self.send_command(f"ninja -C {remote_build_dir}
>> coverage-html", timeout=600)
>> + return True
>> + else:
>> + self._logger.info(
>> + "Unable to generate code coverage report, ensure lcov
>> v1.15 and at least gcov v8.0"
>> + )
>> + return False
>> +
>> def get_dpdk_version(self, build_dir: str | PurePath) -> str:
>> """Overrides :meth:`~.os_session.OSSession.get_dpdk_version`."""
>> out = self.send_command(f"cat {self.join_remote_path(build_dir,
>> 'VERSION')}", verify=True)
>> diff --git a/dts/framework/utils.py b/dts/framework/utils.py
>> index 9917ffbfaa..38da88cd9c 100644
>> --- a/dts/framework/utils.py
>> +++ b/dts/framework/utils.py
>> @@ -125,6 +125,14 @@ def __str__(self) -> str:
>> """The actual args."""
>> return " ".join(f"{self._default_library}
>> {self._dpdk_args}".split())
>>
>> + def _add_arg(self, arg: str):
>> + """Used to add a meson build argument to the DPDK build.
>>
>
> Nit but rephrase to "Adds an argument to the Meson setup command"
>
>
>> +
>> + Args:
>> + arg: The meson build argument to be added.
>> + """
>> + self._dpdk_args = self._dpdk_args + " " + arg
>> +
>>
>> class TarCompressionFormat(StrEnum):
>> """Compression formats that tar can use.
>> --
>> 2.54.0
>>
>>
> Reviewed-by: Patrick Robb <patrickrobb1997 at gmail.com>
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mails.dpdk.org/archives/dev/attachments/20260625/ce8fb8ac/attachment-0001.htm>
More information about the dev
mailing list