[PATCH v8 00/18] Simplify running with high-numbered CPUs

[Bruce Richardson]

The ultimate of this patchset is to make it easier to run on systems
with large numbers of cores, by simplifying the process of using core
numbers >RTE_MAX_LCORE. The new EAL arg ``-remap-lcore-ids``, also
shortened to ``-R``, is added to DPDK to support this.

However, in order to add this new flag easily, the first dozen or more
patches rework the argument handling in EAL to simplify things, using
the argparse library for argument handling.

When processing cmdline arguments in DPDK, we always do so with very
little context. So, for example, when processing the "-l" flag, we have
no idea whether there will be later a --proc-type=secondary flag. We
have all sorts of post-arg-processing checks in place to try and catch
these scenarios.

To improve this situation, this patchset tries to simplify the handling
of argument processing, by explicitly doing an initial pass to collate
all arguments into a structure. Thereafter, the actual arg parsing is
done in a fixed order, meaning that e.g. when processing the
--main-lcore flag, we have already processed the service core flags. We
also can far quicker and easier check for conflicting options, since
they can all be checked for NULL/non-NULL in the arg structure
immediately after the struct has been populated.

An additional benefit of this work is that the argument parsing for EAL
is much more centralised into common options and the options list file.
This single list with ifdefs makes it clear to the viewer what options
are common across OS's, vs what are unix-only or linux-only.

Once the cleanup and rework is done, adding the new options for
remapping cores becomes a lot simpler, since we can very easily check
for scenarios like multi-process and handle those appropriately.

V8:
* dropped the final two patches from the series, dropping the new -L
  option in favour of the -R modifier.
* reordered patch 11 to be with the other argparse patches (now patch 5)
* added patch 12, which uses macros to initialize the args structure
  from the arguments header file, avoiding potential issues when we add
  new args.
* simplified and consolidated lcore mask and core list parsing to always
  work off cpusets rather than arrays of uint8
* enhanced debug printouts to also work better with cpusets and handle
  core values in those sets >= RTE_MAX_LCORE
* for completeness, ensure the new -R option works for coremasks, and
  for cases where no explicit core-list or coremask is specified.

V7:
* expand the scope of the patchset beyond just cleanup to add in the
  extra 3 patches for -L and --lcoreid-base option.

V6:
* Rebase to apply cleanly on latest main

V5:
* Resubmit of V4, which didn't get sent correctly, or picked up correctly
  in patchwork.

V4:
* Updated patch 5 to auto-generate the arg struct definition from the same
  list of defines used to construct the argument list.

V3:
* Added 3 new initial patches, one for minor build-system addition, and two
  for functionality in argparse to allow the user-callback help function to
  be maintained as we move to argparse.
* Added doc updates in the first EAL patch adding the long options
* Fixed ASAN issues by adding a patch to properly clean up EAL init - both
  memory allocations and fixing the run-once flag
* Put ifdefs around the linux-only or unix-only options in EAL patch 2
* Updated args to handle numa-mem and numa-limit as equivalent socket-mem
  and socket-limit


Bruce Richardson (18):
  build: add define for the OS environment name
  argparse: export function to print help text for object
  argparse: allow user-override of help printing
  argparse: add documentation on supported value types
  argparse: add support for parsing core lists
  eal: add long options for each short option
  eal: define the EAL parameters in argparse format
  eal: gather EAL args before processing
  eal: ensure proper cleanup on EAL init failure
  eal: combine parameter validation checks
  eal: simplify handling of conflicting cmdline options
  eal: automatically init arg list options
  eal: add internal fn for converting cpuset to string
  eal: use common cpuset to string function
  eal: introduce lcore remapping option for coremasks
  eal: rework internal coremask parsing to use cpu sets
  eal: allow lcore id remapping with core lists
  eal: allow lcore remapping with autodetected core affinity

 app/test/test_argparse.c                   |  188 +++
 config/meson.build                         |    1 +
 doc/guides/contributing/design.rst         |    2 +-
 doc/guides/linux_gsg/eal_args.include.rst  |   42 +-
 doc/guides/prog_guide/argparse_lib.rst     |  121 +-
 doc/guides/rel_notes/release_25_11.rst     |   18 +
 drivers/event/dlb2/dlb2_priv.h             |    2 +-
 drivers/event/dlb2/pf/base/dlb2_resource.c |   11 +-
 lib/argparse/rte_argparse.c                |  109 +-
 lib/argparse/rte_argparse.h                |   23 +-
 lib/eal/common/eal_common_lcore.c          |   13 +-
 lib/eal/common/eal_common_memory.c         |    3 +-
 lib/eal/common/eal_common_options.c        | 1417 +++++++++++---------
 lib/eal/common/eal_common_thread.c         |   43 +-
 lib/eal/common/eal_option_list.h           |   98 ++
 lib/eal/common/eal_options.h               |  104 +-
 lib/eal/common/eal_private.h               |   21 +
 lib/eal/common/eal_thread.h                |   13 +-
 lib/eal/freebsd/eal.c                      |  245 +---
 lib/eal/include/rte_eal.h                  |   20 -
 lib/eal/linux/eal.c                        |  470 +------
 lib/eal/linux/eal_memory.c                 |    2 +-
 lib/eal/meson.build                        |    2 +-
 lib/eal/windows/eal.c                      |  156 +--
 lib/meson.build                            |    1 +
 25 files changed, 1533 insertions(+), 1592 deletions(-)
 create mode 100644 lib/eal/common/eal_option_list.h

--
2.48.1