[RFC] eal: RFC to refactor rte_eal_init into sub-functions

[Rahul Gupta]

From: Rahul Gupta <rahulgupt at microsoft.com>

Initialization often requires rte_eal_init + rte_pktmbuf_pool_create
which can consume a total time of 500-600 ms:
a) For many devices FLR may take a significant chunk of time
   (200-250 ms in our use-case), this FLR is triggered during device
   probe in rte_eal_init().
b) rte_pktmbuf_pool_create() can consume upto 300-350 ms for
applications that require huge memory.

This cost is incurred on each restart (which happens in our use-case
during binary updates for servicing).
This patch provides an optimization using pthreads that appplications
can use and which can save 200-230ms.

In this patch, rte_eal_init() is refactored into two parts-
a) 1st part is dependent code ie- it’s a perquisite of the FLR and
   mempool creation. So this code needs to be executed before any
   pthreads. Its named as rte_eal_init_setup()
b) 2nd part of code is independent code ie- it can execute in parallel
   to mempool creation in a pthread. Its named as rte_probe_and_ioctl().

Existing applications require no changes unless they wish to leverage
the optimization.

If the application wants to use pthread functionality, it should call-
a) rte_eal_init_setup() then create two or more pthreads-
b) in one pthread call- rte_probe_and_ioctl(),
c) second pthread call- rte_pktmbuf_pool_create()
d) (optional) Other pthreads for  any other independent function.

Signed-off-by: Rahul Gupta <rahulgupt at linux.microsoft.com>
---
 lib/librte_eal/common/include/rte_eal.h |  2 ++
 lib/librte_eal/linux/eal/eal.c          | 18 ++++++++++++++++--
 lib/librte_eal/rte_eal_version.map      |  2 ++
 3 files changed, 20 insertions(+), 2 deletions(-)

diff --git a/lib/librte_eal/common/include/rte_eal.h b/lib/librte_eal/common/include/rte_eal.h
index 2f9ed29..90db16b 100644
--- a/lib/librte_eal/common/include/rte_eal.h
+++ b/lib/librte_eal/common/include/rte_eal.h
@@ -129,6 +129,8 @@ enum rte_proc_type_t {
  *     ENOEXEC indicates that a service core failed to launch successfully.
  */
 int rte_eal_init(int argc, char **argv);
+int rte_eal_init_setup(int argc, char **argv);
+int rte_probe_and_ioctl(void);
 
 /**
  * Clean up the Environment Abstraction Layer (EAL)
diff --git a/lib/librte_eal/linux/eal/eal.c b/lib/librte_eal/linux/eal/eal.c
index 5cabba8..6f2a4d0 100644
--- a/lib/librte_eal/linux/eal/eal.c
+++ b/lib/librte_eal/linux/eal/eal.c
@@ -956,9 +956,8 @@ static void rte_eal_init_alert(const char *msg)
 	return n > 2;
 }
 
-/* Launch threads, called at application init(). */
 int
-rte_eal_init(int argc, char **argv)
+rte_eal_init_setup(int argc, char **argv)
 {
 	int i, fctret, ret;
 	pthread_t thread_id;
@@ -1246,7 +1245,13 @@ static void rte_eal_init_alert(const char *msg)
 	 */
 	rte_eal_mp_remote_launch(sync_func, NULL, SKIP_MASTER);
 	rte_eal_mp_wait_lcore();
+	return fctret;
+}
 
+int
+rte_probe_and_ioctl(void)
+{
+	int ret = 0;
 	/* initialize services so vdevs register service during bus_probe. */
 	ret = rte_service_init();
 	if (ret) {
@@ -1296,7 +1301,16 @@ static void rte_eal_init_alert(const char *msg)
 
 	/* Call each registered callback, if enabled */
 	rte_option_init();
+	return 0;	//return Success if control reaches here
+}
 
+/* Launch threads, called at application init(). */
+int
+rte_eal_init(int argc, char **argv)
+{
+	int fctret = rte_eal_init_setup(argc, argv);
+	if (fctret>=0)
+		fctret = rte_probe_and_ioctl();
 	return fctret;
 }
 
diff --git a/lib/librte_eal/rte_eal_version.map b/lib/librte_eal/rte_eal_version.map
index 6482539..9e6054f 100644
--- a/lib/librte_eal/rte_eal_version.map
+++ b/lib/librte_eal/rte_eal_version.map
@@ -53,6 +53,8 @@ DPDK_20.0 {
 	rte_eal_hotplug_remove;
 	rte_eal_hpet_init;
 	rte_eal_init;
+	rte_eal_init_setup;
+	rte_probe_and_ioctl;
 	rte_eal_iopl_init;
 	rte_eal_iova_mode;
 	rte_eal_lcore_role;
-- 
1.8.3.1