[PATCH v5 00/23] dts: add dts api docs

[Juraj Linkeš]

The commits can be split into groups.

The first commit makes changes to the code. These code changes mainly
change the structure of the code so that the actual API docs generation
works. There are also some code changes which get reflected in the
documentation, such as making functions/methods/attributes private or
public.

The second set of commits (2-21) deal with the actual docstring
documentation (from which the API docs are generated). The format of
docstrings is the Google format [0] with PEP257 [1] and some guidelines
captured in the last commit of this group covering what the Google
format doesn't.
The docstring updates are split into many commits to make review
possible. When accepted, these may be squashed (commits 4-21).
The docstrings have been composed in anticipation of [2], adhering to
maximum line length of 100. We don't have a tool for automatic docstring
formatting, hence the usage of 100 right away to save time.

NOTE: The logger.py module is not fully documented, as it's being
refactored and the refactor will be submitted in the near future.
Documenting it now seems unnecessary.

The last two commits comprise the final group, enabling the actual
generation of documentation.
The generation is done with Sphinx, which DPDK already uses, with
slightly modified configuration (the sidebar: unlimited depth and better
collapsing - I need comment on this).

The first two groups are the most important to merge, as future
development can't proceed without them. The third group may be
finished/accepted at a later date, as it's fairly independent.

The build requires the same Python version and dependencies as DTS,
because Sphinx imports the Python modules. The modules are imported
individually, requiring the code refactoring mentioned above.
Dependencies are installed
using Poetry from the dts directory:

poetry install --with docs

After installing, enter the Poetry shell:

poetry shell

And then run the build:
ninja -C <meson_build_dir> dts-doc

[0] https://google.github.io/styleguide/pyguide.html#s3.8.4-comments-in-classes
[1] https://peps.python.org/pep-0257/
[2] https://patches.dpdk.org/project/dpdk/list/?series=29844

Juraj Linkeš (23):
  dts: code adjustments for doc generation
  dts: add docstring checker
  dts: add basic developer docs
  dts: exceptions docstring update
  dts: settings docstring update
  dts: logger and settings docstring update
  dts: dts runner and main docstring update
  dts: test suite docstring update
  dts: test result docstring update
  dts: config docstring update
  dts: remote session docstring update
  dts: interactive remote session docstring update
  dts: port and virtual device docstring update
  dts: cpu docstring update
  dts: os session docstring update
  dts: posix and linux sessions docstring update
  dts: node docstring update
  dts: sut and tg nodes docstring update
  dts: base traffic generators docstring update
  dts: scapy tg docstring update
  dts: test suites docstring update
  dts: add doc generation dependencies
  dts: add doc generation

 buildtools/call-sphinx-build.py               |  29 +-
 doc/api/meson.build                           |   1 +
 doc/guides/conf.py                            |  34 +-
 doc/guides/meson.build                        |   1 +
 doc/guides/tools/dts.rst                      | 103 ++++
 dts/doc/conf_yaml_schema.json                 |   1 +
 dts/doc/index.rst                             |  17 +
 dts/doc/meson.build                           |  49 ++
 dts/framework/__init__.py                     |  12 +-
 dts/framework/config/__init__.py              | 379 ++++++++++---
 dts/framework/config/types.py                 | 132 +++++
 dts/framework/dts.py                          | 161 +++++-
 dts/framework/exception.py                    | 156 +++---
 dts/framework/logger.py                       |  72 ++-
 dts/framework/remote_session/__init__.py      |  80 ++-
 .../interactive_remote_session.py             |  36 +-
 .../remote_session/interactive_shell.py       | 152 ++++++
 dts/framework/remote_session/os_session.py    | 284 ----------
 dts/framework/remote_session/python_shell.py  |  32 ++
 .../remote_session/remote/__init__.py         |  27 -
 .../remote/interactive_shell.py               | 133 -----
 .../remote_session/remote/python_shell.py     |  12 -
 .../remote_session/remote/remote_session.py   | 172 ------
 .../remote_session/remote/testpmd_shell.py    |  49 --
 .../remote_session/remote_session.py          | 232 ++++++++
 .../{remote => }/ssh_session.py               |  28 +-
 dts/framework/remote_session/testpmd_shell.py |  86 +++
 dts/framework/settings.py                     | 188 +++++--
 dts/framework/test_result.py                  | 296 +++++++---
 dts/framework/test_suite.py                   | 230 ++++++--
 dts/framework/testbed_model/__init__.py       |  28 +-
 dts/framework/testbed_model/{hw => }/cpu.py   | 209 +++++--
 dts/framework/testbed_model/hw/__init__.py    |  27 -
 dts/framework/testbed_model/hw/port.py        |  60 ---
 .../testbed_model/hw/virtual_device.py        |  16 -
 .../linux_session.py                          |  69 ++-
 dts/framework/testbed_model/node.py           | 217 +++++---
 dts/framework/testbed_model/os_session.py     | 425 +++++++++++++++
 dts/framework/testbed_model/port.py           |  93 ++++
 .../posix_session.py                          |  85 ++-
 dts/framework/testbed_model/sut_node.py       | 227 +++++---
 dts/framework/testbed_model/tg_node.py        |  70 +--
 .../testbed_model/traffic_generator.py        |  72 ---
 .../traffic_generator/__init__.py             |  44 ++
 .../capturing_traffic_generator.py            |  52 +-
 .../{ => traffic_generator}/scapy.py          | 114 ++--
 .../traffic_generator/traffic_generator.py    |  87 +++
 dts/framework/testbed_model/virtual_device.py |  29 +
 dts/framework/utils.py                        | 136 +++--
 dts/main.py                                   |  17 +-
 dts/meson.build                               |  16 +
 dts/poetry.lock                               | 509 +++++++++++++++++-
 dts/pyproject.toml                            |  13 +-
 dts/tests/TestSuite_hello_world.py            |  16 +-
 dts/tests/TestSuite_os_udp.py                 |  16 +-
 dts/tests/TestSuite_smoke_tests.py            |  53 +-
 meson.build                                   |   1 +
 57 files changed, 4181 insertions(+), 1704 deletions(-)
 create mode 120000 dts/doc/conf_yaml_schema.json
 create mode 100644 dts/doc/index.rst
 create mode 100644 dts/doc/meson.build
 create mode 100644 dts/framework/config/types.py
 rename dts/framework/remote_session/{remote => }/interactive_remote_session.py (76%)
 create mode 100644 dts/framework/remote_session/interactive_shell.py
 delete mode 100644 dts/framework/remote_session/os_session.py
 create mode 100644 dts/framework/remote_session/python_shell.py
 delete mode 100644 dts/framework/remote_session/remote/__init__.py
 delete mode 100644 dts/framework/remote_session/remote/interactive_shell.py
 delete mode 100644 dts/framework/remote_session/remote/python_shell.py
 delete mode 100644 dts/framework/remote_session/remote/remote_session.py
 delete mode 100644 dts/framework/remote_session/remote/testpmd_shell.py
 create mode 100644 dts/framework/remote_session/remote_session.py
 rename dts/framework/remote_session/{remote => }/ssh_session.py (83%)
 create mode 100644 dts/framework/remote_session/testpmd_shell.py
 rename dts/framework/testbed_model/{hw => }/cpu.py (50%)
 delete mode 100644 dts/framework/testbed_model/hw/__init__.py
 delete mode 100644 dts/framework/testbed_model/hw/port.py
 delete mode 100644 dts/framework/testbed_model/hw/virtual_device.py
 rename dts/framework/{remote_session => testbed_model}/linux_session.py (79%)
 create mode 100644 dts/framework/testbed_model/os_session.py
 create mode 100644 dts/framework/testbed_model/port.py
 rename dts/framework/{remote_session => testbed_model}/posix_session.py (74%)
 delete mode 100644 dts/framework/testbed_model/traffic_generator.py
 create mode 100644 dts/framework/testbed_model/traffic_generator/__init__.py
 rename dts/framework/testbed_model/{ => traffic_generator}/capturing_traffic_generator.py (66%)
 rename dts/framework/testbed_model/{ => traffic_generator}/scapy.py (71%)
 create mode 100644 dts/framework/testbed_model/traffic_generator/traffic_generator.py
 create mode 100644 dts/framework/testbed_model/virtual_device.py
 create mode 100644 dts/meson.build

-- 
2.34.1