[dpdk-dev] [PATCH v5 00/10] Register non-EAL threads as lcore

[David Marchand]

OVS and some other applications have been hacking into DPDK internals to
fake EAL threads and avoid performance penalty of only having non-EAL
threads.

This series proposes to add a new type of lcores and maps those threads
to such lcores.
non-EAL threads won't run the DPDK eal mainloop.
As a consequence, part of the EAL threads API cannot work.

Having new lcores appearing during the process lifetime is not expected
by some DPDK components. This is addressed by introducing init/uninit
callacks invoked when hotplugging of such lcore.

There is still some work/discussion:
- refuse new lcore role in incompatible EAL threads API (or document it
  only as those API were already incompatible?),
- think about deprecation notices for existing RTE_FOREACH_LCORE macros
  and consorts, it is probably worth discussing on how to iterate over
  lcores,

For the interested parties, I have a patch [1] against dpdk-latest OVS
branch that makes use of this series (this patch probably won't work with
v5, it will be rebased once dpdk side is ready).

1: https://patchwork.ozlabs.org/project/openvswitch/patch/20200626123017.28555-1-david.marchand@redhat.com/

Changes since v4:
- added separate API to control mp feature activation,
- addressed Konstantin and Olivier comments,

Changes since v3:
- added init failure when trying to use in conjunction with multiprocess,
- addressed Andrew comments,

Changes since v2:
- fixed windows build error due to missing trace stub,
- fixed bug when rolling back on lcore register,

Changes since v1:
- rebased on master (conflicts on merged Windows series),
- separated lcore role code cleanup in a patch,
- tried to use a single naming, so kept non-EAL threads as the main
  notion. non-EAL threads are then distinguished between registered and
  unregistered non-EAL threads,
- added unit tests (still missing some coverage, marked with a FIXME),
- reworked callbacks call under a common rwlock lock which protects
  lcores allocations and callbacks registration,
- introduced lcore iterators and converted the bucket mempool driver,

-- 
David Marchand
David Marchand (10):
  eal: relocate per thread symbols to common
  eal: fix multiple definition of per lcore thread id
  eal: introduce thread init helper
  eal: introduce thread uninit helper
  eal: move lcore role code
  eal: register non-EAL threads as lcores
  eal: add lcore init callbacks
  eal: add lcore iterators
  mempool/bucket: handle non-EAL lcores
  eal: add multiprocess disable API

 MAINTAINERS                                   |   1 +
 app/test/Makefile                             |   1 +
 app/test/autotest_data.py                     |   6 +
 app/test/meson.build                          |   2 +
 app/test/test_lcores.c                        | 373 ++++++++++++++++++
 app/test/test_mp_secondary.c                  |   7 +
 doc/guides/howto/debug_troubleshoot.rst       |   5 +-
 .../prog_guide/env_abstraction_layer.rst      |  22 +-
 doc/guides/prog_guide/mempool_lib.rst         |   2 +-
 drivers/mempool/bucket/rte_mempool_bucket.c   | 130 +++---
 lib/librte_eal/common/eal_common_lcore.c      | 249 +++++++++++-
 lib/librte_eal/common/eal_common_proc.c       |  41 ++
 lib/librte_eal/common/eal_common_thread.c     | 141 +++++--
 lib/librte_eal/common/eal_common_trace.c      |  49 ++-
 lib/librte_eal/common/eal_memcfg.h            |   2 +
 lib/librte_eal/common/eal_private.h           |  39 ++
 lib/librte_eal/common/eal_thread.h            |  21 +-
 lib/librte_eal/common/eal_trace.h             |   1 +
 lib/librte_eal/freebsd/eal.c                  |  20 +-
 lib/librte_eal/freebsd/eal_thread.c           |  38 +-
 lib/librte_eal/include/rte_eal.h              |  26 +-
 lib/librte_eal/include/rte_lcore.h            | 179 ++++++++-
 lib/librte_eal/linux/eal.c                    |  21 +-
 lib/librte_eal/linux/eal_thread.c             |  38 +-
 lib/librte_eal/rte_eal_version.map            |  16 +
 lib/librte_eal/windows/eal.c                  |   8 +-
 lib/librte_eal/windows/eal_thread.c           |  14 +-
 lib/librte_mempool/rte_mempool.h              |  11 +-
 28 files changed, 1221 insertions(+), 242 deletions(-)
 create mode 100644 app/test/test_lcores.c

-- 
2.23.0