[dpdk-dev] [PATCH v3] eal: add manual probing option

[Jerin Jacob]

On Fri, Oct 4, 2019 at 6:25 PM Gaetan Rivet <gaetan.rivet at 6wind.com> wrote:
>
> Add a new EAL option enabling manual probing in the EAL.
> This command line option will configure the EAL so that buses
> will not trigger their probe step on their own.
>
> Applications are then expected to hotplug devices as they see fit.
>
> Devices declared on the command line by the user (using -w and --vdev),
> will be probed using the hotplug API, in the order they are declared.
>
> This has the effect of offering a way for users to control probe order
> of their devices, for drivers requiring it.
>
> Signed-off-by: Gaetan Rivet <gaetan.rivet at 6wind.com>
> ---
>
> I haven't heard many opinions on the matter, please shout if you see an issue
> with this approach.
>
> @Slava: I have tested rather quickly that it does not break anything,
>         and that it works as intended for basic cases.
>         Can you test it further for your use-case and tell me if it works fine?
>
> Beyond the obvious difference between both probe mode, something to keep in mind:
> while using -w on invalid devices would not block (PCI) bus probing, it will stop manual
> probing in its track. All devices need to exist and be valid device IDs.
>
> v2: fixed a few typos, map file (and used Travis to validate).
>
>     Slava, are you able to test this patch?
>
> v3: properly fixed the map file (herited 19.08 instead of 19.05).
>
>     Added a function to set the probe manual from the application,
>     without having the user do it from the command line.
>
>     Stopped spamming Slava about it, Vamsi was actually the one interested in it!
>
> Standing issue worth chiming in:
>
>   Currently manual-probe will cut off probing from all buses.
>   It could be interesting to be able to only cut buses supporting hotplug,
>   given that they are the one able to probe devices afterward.
>
>   No real use-case for this right now, so leaving as-is. Might be worth
>   considering in the future.
>
>  doc/guides/rel_notes/release_19_11.rst     |  9 +++++++
>  lib/librte_eal/common/eal_common_bus.c     |  6 +++++
>  lib/librte_eal/common/eal_common_dev.c     | 41 ++++++++++++++++++++++++++++++
>  lib/librte_eal/common/eal_common_options.c |  8 ++++++
>  lib/librte_eal/common/eal_internal_cfg.h   |  1 +
>  lib/librte_eal/common/eal_options.h        |  2 ++
>  lib/librte_eal/common/eal_private.h        |  9 +++++++
>  lib/librte_eal/common/include/rte_eal.h    | 34 +++++++++++++++++++++++++
>  lib/librte_eal/freebsd/eal/eal.c           | 10 ++++++++
>  lib/librte_eal/linux/eal/eal.c             | 10 ++++++++
>  lib/librte_eal/rte_eal_version.map         |  8 ++++++
>  11 files changed, 138 insertions(+)
>
> diff --git a/doc/guides/rel_notes/release_19_11.rst b/doc/guides/rel_notes/release_19_11.rst
> index 27cfbd9e3..700f9a726 100644
> --- a/doc/guides/rel_notes/release_19_11.rst
> +++ b/doc/guides/rel_notes/release_19_11.rst
> @@ -56,6 +56,15 @@ New Features
>       Also, make sure to start the actual text at the margin.
>       =========================================================
>
> +* **EAL will now allow manual probing devices.**
> +
> +  Previously, a user could not force an order when probing declared devices.
> +  This could cause issues for drivers depending on another device being present.
> +  A new option ``--manual-probe`` is now available to do just that.
> +  This new option relies on the device bus supporting hotplug. It can
> +  also be used to disable automatic probing from the ``PCI`` bus without
> +  having to disable the whole bus.
> +
>
>  Removed Items
>  -------------
> diff --git a/lib/librte_eal/common/eal_common_bus.c b/lib/librte_eal/common/eal_common_bus.c
> index baa5b532a..145a96812 100644
> --- a/lib/librte_eal/common/eal_common_bus.c
> +++ b/lib/librte_eal/common/eal_common_bus.c
> @@ -6,6 +6,7 @@
>  #include <string.h>
>  #include <sys/queue.h>
>
> +#include <rte_eal.h>
>  #include <rte_bus.h>
>  #include <rte_debug.h>
>  #include <rte_string_fns.h>
> @@ -63,6 +64,11 @@ rte_bus_probe(void)
>         int ret;
>         struct rte_bus *bus, *vbus = NULL;
>
> +       if (rte_eal_manual_probe()) {

See below,



>  int rte_vfio_setup_device(__rte_unused const char *sysfs_base,
>                       __rte_unused const char *dev_addr,
>                       __rte_unused int *vfio_dev_fd,
> diff --git a/lib/librte_eal/linux/eal/eal.c b/lib/librte_eal/linux/eal/eal.c
> index 946222ccd..da00eb14d 100644
> --- a/lib/librte_eal/linux/eal/eal.c
> +++ b/lib/librte_eal/linux/eal/eal.c
> @@ -1382,6 +1382,16 @@ rte_eal_vfio_intr_mode(void)
>         return internal_config.vfio_intr_mode;
>  }
>
> +int rte_eal_manual_probe(void)
> +{
> +       return internal_config.manual_probe;
> +}
> +
> +void rte_eal_manual_probe_set(int enabled)
> +{
> +       internal_config.manual_probe = !!enabled;
> +}

I don't think FreeBSD and Linux specific implementation is NOT required
as internal_config accessible in eal/common. example diff.

[master][dpdk.org] $ git diff
diff --git a/lib/librte_eal/common/eal_common_bus.c
b/lib/librte_eal/common/eal_common_bus.c
index 145a96812..db4257cf1 100644
--- a/lib/librte_eal/common/eal_common_bus.c
+++ b/lib/librte_eal/common/eal_common_bus.c
@@ -13,6 +13,7 @@
 #include <rte_errno.h>

 #include "eal_private.h"
+#include "eal_internal_cfg.h"

 static struct rte_bus_list rte_bus_list =
        TAILQ_HEAD_INITIALIZER(rte_bus_list);
@@ -64,7 +65,7 @@ rte_bus_probe(void)
        int ret;
        struct rte_bus *bus, *vbus = NULL;

-       if (rte_eal_manual_probe()) {
+       if (internal_config.manual_probe) {
                RTE_LOG(DEBUG, EAL, "Manual probing enabled.\n");
                return rte_dev_probe_devargs_list();
        }

>  int
>  rte_eal_check_module(const char *module_name)
>  {
> diff --git a/lib/librte_eal/rte_eal_version.map b/lib/librte_eal/rte_eal_version.map
> index 7cbf82d37..ccc4ffb21 100644
> --- a/lib/librte_eal/rte_eal_version.map
> +++ b/lib/librte_eal/rte_eal_version.map
> @@ -312,6 +312,14 @@ DPDK_19.08 {
>
>  } DPDK_19.05;
>
> +DPDK_19.11 {
> +       global:
> +
> +       rte_eal_manual_probe;
> +       rte_eal_manual_probe_set;

Do we need public API for this? it is only used by eal lib.

checkpatch complains the following too.
ERROR: symbol rte_eal_manual_probe_set is added in the DPDK_19.11
section, but is expected to be added in the EXPERIMENTAL section of
the version map