[dpdk-dev] [PATCH 1/1] eal: Don't fail secondary if primary is missing tailqs

Luca Boccassi bluca at debian.org
Thu Nov 15 10:33:17 CET 2018


On Wed, 2018-11-14 at 18:24 +0000, Burdick, Cliff wrote:
> 
> -----Original Message-----
> From: Luca Boccassi [mailto:bluca at debian.org> Sent: Wednesday, November 14, 2018 10:15 AM
> To: Burdick, Cliff; Bruce Richardson
> Cc: Thomas Monjalon; Burakov, Anatoly; dev at dpdk.org
> Subject: Re: [dpdk-dev] [PATCH 1/1] eal: Don't fail secondary if
> primary is missing tailqs
> 
> > On Wed, 2018-11-14 at 17:40 +0000, Burdick, Cliff wrote:
> > > 
> > > -----Original Message-----
> > > From: Bruce Richardson [mailto:bruce.richardson at intel.com]
> > > Sent: Wednesday, November 14, 2018 3:48 AM
> > > To: Burdick, Cliff
> > > Cc: Thomas Monjalon; Burakov, Anatoly; dev at dpdk.org
> > > Subject: Re: [dpdk-dev] [PATCH 1/1] eal: Don't fail secondary if 
> > > primary is missing tailqs
> > > 
> > > On Tue, Nov 13, 2018 at 11:42:51PM +0000, Burdick, Cliff wrote:
> > > > > 
> > > > > ________________________________________
> > > > > From: Thomas Monjalon [thomas at monjalon.net]
> > > > > Sent: Tuesday, November 13, 2018 2:18 PM
> > > > > To: Burdick, Cliff
> > > > > Cc: Burakov, Anatoly; dev at dpdk.org; bruce.richardson at intel.co
> > > > > m
> > > > > Subject: Re: [dpdk-dev] [PATCH 1/1] eal: Don't fail secondary
> > > > > if 
> > > > > primary is missing tailqs
> > > > > 
> > > > > 13/11/2018 23:08, Burdick, Cliff:
> > > > > > From: Thomas Monjalon [mailto:thomas at monjalon.net]
> > > > > > > 13/11/2018 17:38, Burdick, Cliff:
> > > > > > > > From: Thomas Monjalon [mailto:thomas at monjalon.net]
> > > > > > > > 13/11/2018 16:45, Burdick, Cliff:
> > > > > > > > > From: Burakov, Anatoly [mailto:anatoly.burakov at intel.
> > > > > > > > > com]
> > > > > > > > > > On 13-Nov-18 9:21 AM, Thomas Monjalon wrote:
> > > > > > > > > > > 13/11/2018 00:33, Burdick, Cliff:
> > > > > > > > > > > > This patch was submitted by Jean Tourrilhes
> > > > > > > > > > > > over two 
> > > > > > > > > > > > years ago, but didn't receive any responses. I
> > > > > > > > > > > > hit 
> > > > > > > > > > > > the same issue recently when trying to use cgo 
> > > > > > > > > > > > (Golang) as a primary process linked to
> > > > > > > > > > > > libdpdk.a 
> > > > > > > > > > > > against a C++ application linked against the
> > > > > > > > > > > > same 
> > > > > > > > > > > > library.> > >
> > > > > > > > > > > 
> > > > > > > > > > > The question is to know why you don't have the
> > > > > > > > > > > same 
> > > > > > > > > > > constructors in primary and seconday?
> > > > > > > > > > 
> > > > > > > > > > I've hit similar things in the past. I believe it
> > > > > > > > > > was 
> > > > > > > > > > caused by our build system stripping out unused 
> > > > > > > > > > libraries (such as
> > > > > > > > > > rte_hash) from the binary and thus not calling the 
> > > > > > > > > > constructor in the primary, but doing so in the 
> > > > > > > > > > secondary (or something to that effect). In any
> > > > > > > > > > case, 
> > > > > > > > > > this is caused by linking different number of
> > > > > > > > > > libraries 
> > > > > > > > > > to primary and secondary, and should probably be
> > > > > > > > > > fixed 
> > > > > > > > > > in the build system, not in the tailqs code (unless
> > > > > > > > > > we 
> > > > > > > > > > specifically support having different linked
> > > > > > > > > > libraries 
> > > > > > > > > > to primary and secondary?).> > > >
> > > > > > > > > 
> > > > > > > > > Right, I think the original author of the patch
> > > > > > > > > stated the 
> > > > > > > > > reasons in the link I provided. The build system
> > > > > > > > > seems 
> > > > > > > > > like the most appropriate place to fix it, but the
> > > > > > > > > patch 
> > > > > > > > > got me going quickly. I think the question is whether
> > > > > > > > > you 
> > > > > > > > > want DPDK to support these other ways of linking.
> > > > > > > > > I'm 
> > > > > > > > > certainly not the first to use cgo, since there's a 
> > > > > > > > > virtual switch project doing the same:
> > > > > > > > > 
> > > > > > > > > https://urldefense.proofpoint.com/v2/url?u=https-3A__
> > > > > > > > > gith
> > > > > > > > > ub.co
> > > > > > > > > m_lago
> > > > > > > > > pu
> > > > > > > > > s_vsw&d=DwICAg&c=jcv3orpCsv7C4ly8-
> > > > > > > > > ubDoUfrxF5xIGWmptxGWP5vi5w&r =m1RLQ OG
> > > > > > > > > Okz9MauvVLZmiGtyWc5mA7DejbPFNE1IDj-
> > > > > > > > > 4&m=hQqVCNwW7eoEzB_hLFK97i8 idS8FI qX
> > > > > > > > > oPeclwsIZq7Y&s=BMoBlwkqljwWIBY3SE3pPMCfVnOUlDuZLrno4-
> > > > > > > > > SojKM&e=
> > > > > > > > > 
> > > > > > > > > They don't use primary/secondary processes, though,
> > > > > > > > > so the 
> > > > > > > > > issue is  never hit. I'm in a situation where using
> > > > > > > > > cgo 
> > > > > > > > > seemed like the easiest  path to accomplish what I'm
> > > > > > > > > doing 
> > > > > > > > > since I needed specialized  libraries for it that
> > > > > > > > > were not 
> > > > > > > > > available in C/C++. At some point I  bet someone
> > > > > > > > > would use 
> > > > > > > > > Cython to start linking against DPDK as well,  and
> > > > > > > > > we'd 
> > > > > > > > > likely run into the same issue.> > > > For sure, we
> > > > > > > > > want 
> > > > > > > > > to support using DPDK with cgo or cython.
> > > > > > > > > But it is not clear what is the relation with not
> > > > > > > > > having 
> > > > > > > > > the same compilation for primary and secondary.
> > > > > > > > > Please 
> > > > > > > > > could you elaborate?> > >
> > > > > > > > 
> > > > > > > > Hi Thomas, I think Jean explained it well here:
> > > > > > > > https://urldefense.proofpoint.com/v2/url?u=https-3A__de
> > > > > > > > v.dp
> > > > > > > > dk.narkive.
> > > > > > > > com_ZM3a7QD1_dpdk-2Ddev-2Dbug-2Dstatic-2Dconstructors-
> > > > > > > > 2Dconsider
> > > > > > > > ed-2De
> > > > > > > > vil&d=DwICAg&c=jcv3orpCsv7C4ly8- 
> > > > > > > > ubDoUfrxF5xIGWmptxGWP5vi5w&r=m1R LQOGOk
> > > > > > > > z9MauvVLZmiGtyWc5mA7DejbPFNE1IDj- 
> > > > > > > > 4&m=C69wDgrjDX8_oXp1M_3bnmWOOZd GqwBBG 
> > > > > > > > 9vzkyGDWGQ&s=YYn2N7WrzJpH1ptNrLZad0nPAQdrUyqBckk2uFuWYP
> > > > > > > > Q&e=
> > > > > > > > 
> > > > > > > > "The build system of the application does not have all
> > > > > > > > the 
> > > > > > > > subtelties of the DPDK build system, and ends up
> > > > > > > > including
> > > > > > > > *all*
> > > > > > > > the constructors, wether they are used or not in the
> > > > > > > > code. 
> > > > > > > > Moreover, they are included in a different order.
> > > > > > > > Actually, 
> > > > > > > > by default the builds include no constructors at all
> > > > > > > > (which 
> > > > > > > > is a big fail), so the library needs to be included
> > > > > > > > with 
> > > > > > > > --whole-archive (see Snort DPDK instructions)."
> > > > > > > > 
> > > > > > > > I will get to the bottom of my exact case to
> > > > > > > > understand 
> > > > > > > > what's happening, but my primary application is a cgo 
> > > > > > > > application that I'm linking to by using almost exactly
> > > > > > > > the 
> > > > > > > > same flags that are used in the DPDK build system to
> > > > > > > > build 
> > > > > > > > examples. The DPDK libraries I'm linking against is a
> > > > > > > > single 
> > > > > > > > location for both primary and secondary; in other
> > > > > > > > words, I 
> > > > > > > > don't build DPDK twice.
> > > > > > > > 
> > > > > > > > OK I understand, thanks.
> > > > > > > > 
> > > > > > > > You had alluded to a pkg-config for DPDK in the 2015
> > > > > > > > thread, 
> > > > > > > > which cgo can use, but I don't know if that ever was 
> > > > > > > > implemented. Cgo can use pkg-config if it's available, 
> > > > > > > > otherwise the only tools are specifying LDFLAGS and
> > > > > > > > CFLAGS 
> > > > > > > > into their build system.
> > > > > > > 
> > > > > > > Yes, the right answer is still pkg-config :) The good
> > > > > > > news is 
> > > > > > > that it is now implemented thanks to the meson build
> > > > > > > system:
> > > > > > >     
> > > > > > > https://urldefense.proofpoint.com/v2/url?u=http-3A__git.d
> > > > > > > pdk.
> > > > > > > org_dp
> > > > > > > dk_tree_doc_build-2Dsdk-2Dmeson.txt-
> > > > > > > 23n182&d=DwICAg&c=jcv3orpCsv7C4
> > > > > > > ly8-
> > > > > > > ubDoUfrxF5xIGWmptxGWP5vi5w&r=m1RLQOGOkz9MauvVLZmiGtyWc5mA
> > > > > > > 7Dej
> > > > > > > bP
> > > > > > > FNE1IDj-
> > > > > > > > 4&m=C69wDgrjDX8_oXp1M_3bnmWOOZdGqwBBG9vzkyGDWGQ&s=oC86K
> > > > > > > > D_R
> > > > > > > 
> > > > > > > J1T6rfzi3x5zFT1Ri13ELpKmsyFqpgDbgFg&e=
> > > > > > 
> > > > > > Hi Thomas, are there instructions on how to use pkg-config
> > > > > > to 
> > > > > > build the mlx4/5 PMD? I noticed a patch was submitted in 
> > > > > > September to add support for it, but that link you provided
> > > > > > on 
> > > > > > using meson doesn't say how to build specific drivers. It 
> > > > > > appears to be disabled by default.
> > > > > > If the dependency is found, meson will enable the PMD when 
> > > > > > building DPDK.
> > > > > 
> > > > > Do you know where exactly that is? I've been using mlx5 for
> > > > > a 
> > > > > while on this system, and I can see that 18.11-rc2 meson
> > > > > build+ninja built the pmd, but it's not on the --libs listing
> > > > > for
> > > > > pkg-config. Does it tell me what I was missing?
> > > > > 
> > > > 
> > > > For dynamic linking of applications, the drivers are not
> > > > normally 
> > > > linked in. Instead, they should be loaded from the drivers
> > > > directory 
> > > > as .so files
> > > > - normally by default in EAL as the driver .so's should be
> > > > copied to 
> > > > the EAL_PMD_PATH where EAL finds them automatically. [This
> > > > applies 
> > > > to both meson and make builds, though only meson generates the
> > > > .pc 
> > > > file for pkg-config]
> > > > 
> > > > If you are doing a static build, then you need to explicitly
> > > > link in 
> > > > the drivers. You can get a list from pkg-config using the "
> > > > --static" flag in this case. A good example of how to use pkg- 
> > > > config in this way can be found in the Makefiles for most
> > > > examples, 
> > > > e.g. examples/helloworld/Makefile, reproduced below.
> > > > 
> > > > Regards,
> > > > /Bruce
> > > > 
> > > > all: shared
> > > > .PHONY: shared static
> > > > shared: build/$(APP)-shared
> > > >         ln -sf $(APP)-shared build/$(APP)
> > > > static: build/$(APP)-static
> > > >         ln -sf $(APP)-static build/$(APP)
> > > > 
> > > > PC_FILE := $(shell pkg-config --path libdpdk) CFLAGS += -O3
> > > > $(shell 
> > > > pkg-config --cflags libdpdk) LDFLAGS_SHARED = $(shell pkg-
> > > > config -- 
> > > > libs libdpdk) LDFLAGS_STATIC = -Wl,-Bstatic $(shell pkg-config
> > > > -- 
> > > > static --libs libdpdk)
> > > > 
> > > > build/$(APP)-shared: $(SRCS-y) Makefile $(PC_FILE) | build
> > > >         $(CC) $(CFLAGS) $(SRCS-y) -o $@ $(LDFLAGS)
> > > > $(LDFLAGS_SHARED)
> > > > 
> > > > build/$(APP)-static: $(SRCS-y) Makefile $(PC_FILE) | build
> > > >         $(CC) $(CFLAGS) $(SRCS-y) -o $@ $(LDFLAGS)
> > > > $(LDFLAGS_STATIC)
> > > > 
> > > > build:
> > > >         @mkdir -p $@
> > > 
> > > Thanks Bruce. I tried getting this to compile using cgo, and
> > > it's 
> > > causing more pain than it's worth. I used the --static --libs
> > > options, 
> > > and there were still numerous linker errors, some specific to
> > > mlx, and 
> > > some not. Specifically libmlx5 and libmnl are both needed, but
> > > they're 
> > > not part of the linker line from pkg-config. At this point I'll
> > > just 
> > > break the Go application up into a separate C application that
> > > handles 
> > > all the DPDK parts, and send messages between them.
> > 
> > Hi,
> > 
> > As far as I can see, both mnl and mlx5 (and all the other reverse
> > dependencies) are listed correctly in the libdpdk.pc Libs.private
> > entry, and pkg-config --libs --static libdpdk.pc does print them as
> > intended. This is with 18.11-rc3.
> > Are you sure everything is correct (pkg-config path is right, --
> > static is used, etc)?
> > 
> > --
> > Kind regards,
> > Luca Boccassi
> 
> Hi Luca, here is what pkg-config gives:
> 
> PKG_CONFIG_PATH=meson-private/ pkg-config --libs --static libdpdk
> -L/usr/local/lib/x86_64-linux-gnu -lrte_eventdev -lrte_power
> -lrte_ip_frag -lrte_hash -lrte_pdump -lrte_eal -lrte_pipeline
> -lrte_table -lrte_bbdev -lrte_bpf -lrte_vhost -lrte_metrics
> -lrte_jobstats -lrte_net -lrte_reorder -lrte_ring -lrte_mempool
> -lrte_security -lrte_compressdev -lrte_rawdev -lrte_mbuf -lrte_kvargs
> -lrte_port -lrte_pci -lrte_ethdev -lrte_gro -lrte_gso -lrte_cryptodev
> -lrte_sched -lrte_latencystats -lrte_efd -lrte_distributor
> -lrte_meter -lrte_acl -lrte_member -lrte_cmdline -lrte_lpm -lrte_kni
> -lrte_cfgfile -lrte_bitratestats -lrte_timer -lrte_flow_classify
> -lrte_mempool_bucket -lrte_pmd_null_crypto -lrte_pmd_failsafe
> -lrte_bus_ifpga -lrte_pmd_atlantic -lrte_pmd_mlx4 -lrte_pmd_vmxnet3
> -lrte_pmd_avp -lrte_common_dpaax -lrte_pmd_ena -lcrypto
> -lrte_bus_fslmc -ldl -lrte_pmd_avf -lrte_pmd_crypto_scheduler
> -lrte_pmd_netvsc -lrte_pmd_vhost -lrte_bus_dpaa -lrte_mempool_dpaa2
> -lrte_common_octeontx -lrte_pmd_liquidio -lrte_pmd_ifc -Wl,-Bdynamic
> -lnuma -lrte_pmd_enic -lrte_common_cpt -pthread
> -lrte_pmd_octeontx_crypto -lrte_pmd_qat -lrte_bus_vmbus
> -lrte_pmd_null -lm -lrte_pmd_dpaa -lrte_bus_vdev -lrte_pmd_dpaa2
> -lrte_pmd_skeleton_event -lrte_pmd_af_packet -lrte_pmd_octeontx
> -lrte_pmd_thunderx -lrte_pmd_ring -lrte_pmd_octeontx_event
> -lrte_pmd_sw_event -lrte_pmd_ark -lrte_mempool_octeontx
> -lrte_pmd_ixgbe -lrte_pmd_kni -lrte_mempool_ring
> -lrte_pmd_virtio_crypto -lrte_mempool_dpaa -lrte_pmd_dpaa2_cmdif
> -lrte_bus_pci -lrte_pmd_opdl_event -lrte_pmd_mlx5 -lrte_pmd_virtio
> -lrte_pmd_tap -lrte_pmd_caam_jr -lrte_pmd_dpaa2_sec
> -lrte_pmd_dpaa2_qdma -lrte_pmd_enetc -lrte_pmd_sfc -lrte_pmd_bnxt
> -lrte_pmd_dpaa2_event -lrte_pmd_cxgbe -Wl,--whole-archive -Wl,--no-
> as-needed -lrte_pmd_e1000 -lrte_pmd_softnic -lrte_pmd_bond
> -lrte_pmd_fm10k -lrte_pmd_i40e -lrte_pmd_nfp -lrte_pmd_qede -Wl,--no-
> whole-archive -lrte_pmd_axgbe -lrte_pmd_ccp -lrte_pmd_ifpga_rawdev
> -lrte_pmd_bbdev_null -lrte_pmd_openssl -lrte_pmd_dsw_event
> -lrte_pmd_dpaa_event -lrte_mempool_stack -lrte_pmd_vdev_netvsc
> -lrte_pmd_dpaa_sec -lrte_pmd_skeleton_rawdev
> -lrte_pmd_octeontx_compress
> 
> You'll notice there's no mlx5 or mnl in that list. I was using 18.11-
> rc2 since I cloned early yesterday. Perhaps meson determined not to
> put it in there for some reason?

Were mlx5/mnl found at meson configure time?

-- 
Kind regards,
Luca Boccassi


More information about the dev mailing list