[RFC PATCH 00/12] Add config-time dependency tracking

[Bruce Richardson]

Add configuration time dependency tracking for libs and drivers in DPDK,
so that when a user requests a driver, all dependent drivers also get
enabled. Similarly with libraries. What is not enabled in this RFC is
support for enabling libraries based on driver requests, or enabling
either libs or drivers based on the enable-apps config option. I've
omitted the former for now, because it would involve moving more logic
out of the lib/meson.build file and drivers/meson.build files into the
top-level one, something I'm a little hesitant to do for now. The
addition of apps only adds value once the cross-type dependency tracking
is enabled, since no app depends on any others.

The first 5 patches in this series are cleanups and bug fixes. The
following patches then implement the dependency tracking. While I
investigated having meson parse each components meson.build file
initially, without actually building it, this failed to work due to
logic other than variable assignments in many drivers.

Instead, python scripts are used to manage the dependency list. For
easier debug, and future introspection this dependency management is
split between two scripts - the first generates a dependency table file
which it writes to the build folder, allowing us to analyse this
post-build if so desired. The second script uses this file to actually
compute the necessary per-component dependencies.

The final patch of the series is a major reduction in the number of
required components for doing DPDK builds. We completely remove the list
of required libraries, and assume that a minimal build should only
require enough libraries to build EAL. However, EAL does assume that we
have a few additional buses, for example, EAL args are present for pci
and vdev. Also, in order for mempools to work, we always need to have
the default mempool ring driver build. That means that our de-facto
minimal build target set is the set of components needed to build
"bus/pci,bus/vdev,mempool/ring" (which would include EAL in the output,
which is why we can remove any hard-coded library lists)

Bruce Richardson (12):
  build: fix name of base library object
  build: fix ignoring requested driver when class is disabled
  build: prevent overlap of enable and disable lists
  build: add direct support for optional dependencies
  net/bond: rename directory for consistency
  build: add script to generate dependencies list
  build: use recursive addition for enabled libraries option
  build: use recursive addition for enabled drivers option
  build: improve error reporting for unbuildable components
  build: add EAL common directory to global include list
  build: make EAL an explicit dependency
  build: simplify mandatory library lists

 MAINTAINERS                                   |   2 +-
 app/meson.build                               |  15 +-
 app/proc-info/meson.build                     |   4 +-
 app/test-bbdev/meson.build                    |  18 +-
 app/test-crypto-perf/meson.build              |   4 +-
 app/test-pmd/meson.build                      |  50 ++---
 app/test/meson.build                          |   6 -
 buildtools/get-deps.py                        | 203 ++++++++++++++++++
 buildtools/meson.build                        |   2 +
 buildtools/resolve-deps.py                    |  80 +++++++
 devtools/test-meson-builds.sh                 |   3 +-
 doc/api/doxy-api.conf.in                      |   2 +-
 .../link_bonding_poll_mode_drv_lib.rst        |   2 +-
 doc/guides/testpmd_app_ug/testpmd_funcs.rst   |   2 +-
 drivers/common/mlx5/linux/meson.build         |   1 -
 drivers/meson.build                           |  85 ++++++--
 .../net/{bonding => bond}/bonding_testpmd.c   |   0
 .../eth_bond_8023ad_private.h                 |   0
 .../net/{bonding => bond}/eth_bond_private.h  |   0
 drivers/net/{bonding => bond}/meson.build     |   1 -
 drivers/net/{bonding => bond}/rte_eth_bond.h  |   0
 .../{bonding => bond}/rte_eth_bond_8023ad.c   |   0
 .../{bonding => bond}/rte_eth_bond_8023ad.h   |   0
 .../net/{bonding => bond}/rte_eth_bond_alb.c  |   0
 .../net/{bonding => bond}/rte_eth_bond_alb.h  |   0
 .../net/{bonding => bond}/rte_eth_bond_api.c  |   0
 .../net/{bonding => bond}/rte_eth_bond_args.c |   0
 .../net/{bonding => bond}/rte_eth_bond_flow.c |   0
 .../net/{bonding => bond}/rte_eth_bond_pmd.c  |   0
 drivers/net/intel/ice/meson.build             |   7 +-
 drivers/net/meson.build                       |   2 +-
 lib/acl/meson.build                           |   1 +
 lib/cfgfile/meson.build                       |   1 +
 lib/cryptodev/meson.build                     |   2 +-
 lib/dmadev/meson.build                        |   2 +-
 lib/eal/meson.build                           |   6 +-
 lib/ethdev/meson.build                        |   2 +-
 lib/eventdev/meson.build                      |   1 -
 lib/ipsec/meson.build                         |   2 +-
 lib/jobstats/meson.build                      |   1 +
 lib/log/meson.build                           |   1 -
 lib/mempool/meson.build                       |   2 +-
 lib/meson.build                               |  76 +++----
 lib/meter/meson.build                         |   1 +
 lib/metrics/meson.build                       |   2 +-
 lib/pci/meson.build                           |   1 +
 lib/ptr_compress/meson.build                  |   1 +
 lib/rawdev/meson.build                        |   2 +-
 lib/ring/meson.build                          |   2 +-
 lib/stack/meson.build                         |   1 +
 lib/timer/meson.build                         |   1 +
 meson.build                                   |  21 ++
 52 files changed, 478 insertions(+), 140 deletions(-)
 create mode 100644 buildtools/get-deps.py
 create mode 100644 buildtools/resolve-deps.py
 rename drivers/net/{bonding => bond}/bonding_testpmd.c (100%)
 rename drivers/net/{bonding => bond}/eth_bond_8023ad_private.h (100%)
 rename drivers/net/{bonding => bond}/eth_bond_private.h (100%)
 rename drivers/net/{bonding => bond}/meson.build (95%)
 rename drivers/net/{bonding => bond}/rte_eth_bond.h (100%)
 rename drivers/net/{bonding => bond}/rte_eth_bond_8023ad.c (100%)
 rename drivers/net/{bonding => bond}/rte_eth_bond_8023ad.h (100%)
 rename drivers/net/{bonding => bond}/rte_eth_bond_alb.c (100%)
 rename drivers/net/{bonding => bond}/rte_eth_bond_alb.h (100%)
 rename drivers/net/{bonding => bond}/rte_eth_bond_api.c (100%)
 rename drivers/net/{bonding => bond}/rte_eth_bond_args.c (100%)
 rename drivers/net/{bonding => bond}/rte_eth_bond_flow.c (100%)
 rename drivers/net/{bonding => bond}/rte_eth_bond_pmd.c (100%)

--
2.53.0