[dpdk-users] Problems linking DPDK-20.11 libraries to another program

[Templin (US), Fred L]

Hello, I am at an impasse trying to link DPDK-20.11 libraries with another program.
The other program (call it "testprog") is a large C source code application that uses
standard linux open source build and compile facilities, including Makefiles, gcc,
configure, etc. It is built from about 190 *.[ch] files, with one defining a single
main() for the entire application.

I read the DPDK documentation and tried to link the libraries in using the
"pkg-config" facility per the recommendations, but I was unable to get pkg-config
to execute out of the "testprog" Makefile. So, I issued the following commands to
obtain the results of running pkg-config:

fltemplin at aer-dtn:~$ pkg-config --cflags libdpdk
fltemplin at aer-dtn:~$ pkg-config --libs libdpdk

I then added the results of the commands to the "testprog" Makefile by adding
the following lines:

CFLAGS += -include rte_config.h -march=native -I/usr/local/include
LDFLAGS += -L/usr/local/lib/x86_64-linux-gnu -Wl,--as-needed -lrte_node -lrte_graph -lrte_bpf -lrte_flow_classify -lrte_pipeline -lrte_table -lrte_port -lrte_fib -lrte_ipsec -lrte_vhost -lrte_stack -lrte_security -lrte_sched -lrte_reorder -lrte_rib -lrte_regexdev -lrte_rawdev -lrte_pdump -lrte_power -lrte_member -lrte_lpm -lrte_latencystats -lrte_kni -lrte_jobstats -lrte_ip_frag -lrte_gso -lrte_gro -lrte_eventdev -lrte_efd -lrte_distributor -lrte_cryptodev -lrte_compressdev -lrte_cfgfile -lrte_bitratestats -lrte_bbdev -lrte_acl -lrte_timer -lrte_hash -lrte_metrics -lrte_cmdline -lrte_pci -lrte_ethdev -lrte_meter -lrte_net -lrte_mbuf -lrte_mempool -lrte_rcu -lrte_ring -lrte_eal -lrte_telemetry -lrte_kvargs

Next, I modified the DPDK "helloworld" example main.c to link it into "testprog" by
changing the name to "hello.c" and changing the "main()" declaration to a function
call called: "testprog_hello()" which I then called from one of my other "testprog"
C modules. The diffs for how I modified the helloworld code appear at the end of
this message.

I was then able to run "make" in the "testprog" directory, and the build compiled
and appeared to link everything just fine. However, it appears that the resulting
"tesprog" binary only increased in size by about 4% over the size it normally builds
to without the DPDK libraries. This seems at odds with the fact that the DPDK
"dpdk-helloworld" binary is massively large in comparison especially considering
that the helloworld main.c is only 47 lines of C code! The sizes of the binaries
are shown below:

fltemplin at aer-dtn:~$ cd ~/src/testprog/build
fltemplin at aer-dtn:~/src/testprog/build$ ls -l testprog*
-rwxr-xr-x 1 fltemplin fltemplin 4991800 Apr 16 10:25 testprog.nodpdk
-rwxr-xr-x 1 fltemplin fltemplin 5186816 Apr 16 10:32 testprog.withdpdk
fltemplin at aer-dtn~/src/testpog/build$: cd ~/src/DPDK/dpdk-20.11/build/examples/
fltemplin at aer-dtn:~/src/DPDK/dpdk-20.11/build/examples$ ls -l dpdk-helloworld
-rwxr-xr-x 1 fltemplin fltemplin 17834608 Apr 14 14:30 dpdk-helloworld

The next question then came to whether my combined testprog/dpdk code
would run the same as the native DPDK helloworld example. I ultimately want
to set up a NULL network interface when I call "rte_eal_init (argc, argv)" and
so I tried issuing the following invocation of dpdk-helloworld:

fltemplin at aer-dtn:~$ sudo ./dpdk-helloworld --vdev net_null0
EAL: Detected 1 lcore(s)
EAL: Detected 1 NUMA nodes
EAL: Multi-process socket /var/run/dpdk/rte/mp_socket
EAL: Selected IOVA mode 'PA'
EAL: Probing VFIO support...
EAL: VFIO support initialized
EAL:   Invalid NUMA socket, default to 0
EAL:   Invalid NUMA socket, default to 0
EAL: No legacy callbacks, legacy socket not created
hello from core 0

So, as you can see, dpdk-helloworld passes the "--vnet" and "net_null0" arguments
to "rte_eal_init()", which recognizes the arguments and returns success. But, when
I call "rte_eal_init()" out of my testprog using the same arguments the DPDK service
returns the following:

fltemplin at aer-dtn:~$ sudo ./testprog
EAL: Detected 1 lcore(s)
EAL: Detected 1 NUMA nodes
EAL: failed to parse device "net_null0"
EAL: Unable to parse device 'net_null0'
PANIC in testprog_hello():
Cannot init EAL

This appears to me that DPDK is failing to initialize basic data structures
when run out of my testprog that are initialized fine when run out of the
dpdk-helloworld binary. That the code compiles and links is encouraging,
but the fact that the binary size for my combined testprog/helloworld is
significantly smaller than the size of dpdk-helloworld alone seems very
suspicious. I feel I must be missing something very basic - does anyone
recognize these conditions and have insights about how to fix?

Thanks - Fred

As previously mentioned, see below for my source code changes to link
helloworld into testprog:

*** ~/src/DPDK/dpdk-20.11/examples/helloworld/main.c	2021-04-14 13:59:16.224073175 -0700
--- ~/src/testprog/src/hello.c	2021-04-16 09:46:42.142210166 -0700
***************
*** 2,7 ****
--- 2,14 ----
   * Copyright(c) 2010-2014 Intel Corporation
   */
  
+ #ifdef HAVE_CONFIG_H
+ #include "config.h"
+ #elif defined(_MSC_VER)
+ #include "config-msvc.h"
+ #endif
+ 
+ #ifdef ENABLE_DPDK_TEST
  #include <stdio.h>
  #include <string.h>
  #include <stdint.h>
***************
*** 25,32 ****
--- 32,47 ----
  }
  
  int
+ #ifdef ENABLE_DPDK_TEST
+ testprog_hello()
+ {
+ 	char *args[] = { "openvpn", "--vdev", "net_null0" };
+ 	char **argv = args;
+ 	int argc = 3;
+ #else
  main(int argc, char **argv)
  {
+ #endif
  	int ret;
  	unsigned lcore_id;
  
***************
*** 45,47 ****
--- 60,63 ----
  	rte_eal_mp_wait_lcore();
  	return 0;
  }
+ #endif