[dpdk-dev] [EXT] Re: performance degradation with fpic

[Jerin Jacob Kollanukkaran]

> -----Original Message-----
> From: Thomas Monjalon <thomas at monjalon.net>
> Sent: Thursday, October 15, 2020 10:45 PM
> To: Ali Alnubani <alialnu at nvidia.com>; Bruce Richardson
> <bruce.richardson at intel.com>
> Cc: dev at dpdk.org; Asaf Penso <asafp at nvidia.com>;
> david.marchand at redhat.com; arybchenko at solarflare.com;
> ferruh.yigit at intel.com; honnappa.nagarahalli at arm.com; Jerin Jacob
> Kollanukkaran <jerinj at marvell.com>
> Subject: [EXT] Re: performance degradation with fpic
> 
> External Email
> 
> ----------------------------------------------------------------------
> 15/10/2020 19:08, Bruce Richardson:
> > On Thu, Oct 15, 2020 at 04:00:44PM +0000, Ali Alnubani wrote:
> > >    We have been seeing in some cases that the DPDK forwarding
> performance
> > >    is up to 9% lower when DPDK is built as static with meson compared to a
> > >    build with makefiles.
> > >
> > >    The same degradation can be reproduced with makefiles on older DPDK
> > >    releases when building with EXTAR_CFLAGS set to “-fPIC”, it can also be
> > >    resolved in meson when passing “pic: false” to meson’s static_library
> > >    call (more tweaking needs to be done to prevent building shared
> > >    libraries because this change breaks them).
> [...]
> > >    Should we disable PIC in static builds?
> >
> > thanks for reporting, though it's strange that you see such a big impact.
> > In my previous tests with i40e driver I never noticed a difference
> > between make and meson builds, and I and some others here have been
> > using meson builds for any performance work for over a year now. That
> > being said let me reverify what I see on my end.
> >
> > In terms of solutions, disabling the -fPIC flag globally implies that
> > we can no longer build static and shared libs from the same sources,
> > so we would need to revert to doing either a static or a shared
> > library build but not both. If the issue is limited to only some
> > drivers or some cases, we can perhaps add in a build option to have
> > no-fpic-static builds, to be used in a cases where it is problematic.

We have seen this issue earlier. Our issue was, meson, getting more performance
Than make build system(Based on different changeset). Initially we suspected fpic
is playing role. Based on our understanding, It not is fpic issue per say, it is more
of text section  code alignment change was creating the issue.
Typically it happen with very "fine" grained, prefetches in Rx and Tx routines, then
All timing will get changed by radical change to text section by fpic.

> 
> I assume only some Rx/Tx functions are impacted.
> We probably need such disabling option per-file.
> 
> > However, at this point, I think we need a little more investigation.
> > Is there any testing you can do to see if it's just in your driver, or
> > in perhaps a mempool driver/lib that the issue appears, or if it's
> > just a global slowdown? Do you see the impact with both clang and gcc?
> > I'll retest things a bit tomorrow on my end to see what I see.
> 
> Yes we need to know which libs or files are impacted by -fPIC.
>