[dpdk-dev] [PATCH v16 1/3] build: disable/enable drivers in Arm builds

[Juraj Linkeš]

> -----Original Message-----
> From: Bruce Richardson <bruce.richardson at intel.com>
> Sent: Wednesday, March 31, 2021 11:08 AM
> To: Juraj Linkeš <juraj.linkes at pantheon.tech>
> Cc: Honnappa Nagarahalli <Honnappa.Nagarahalli at arm.com>; Jerin Jacob
> <jerinjacobk at gmail.com>; Ruifeng Wang <Ruifeng.Wang at arm.com>;
> vcchunga at amazon.com; Dharmik Thakkar <Dharmik.Thakkar at arm.com>;
> hemant.agrawal at nxp.com; Ajit Khaparde (ajit.khaparde at broadcom.com)
> <ajit.khaparde at broadcom.com>; ferruh.yigit at intel.com; aboyer at pensando.io;
> lironh at marvell.com; dev at dpdk.org; nd <nd at arm.com>
> Subject: Re: [PATCH v16 1/3] build: disable/enable drivers in Arm builds
> 
> On Wed, Mar 31, 2021 at 08:39:57AM +0000, Juraj Linkeš wrote:
> >
> >
> > > -----Original Message-----
> > > From: Honnappa Nagarahalli <Honnappa.Nagarahalli at arm.com>
> > > Sent: Tuesday, March 30, 2021 2:39 AM
> > > To: Jerin Jacob <jerinjacobk at gmail.com>; Juraj Linkeš
> > > <juraj.linkes at pantheon.tech>
> > > Cc: Bruce Richardson <bruce.richardson at intel.com>; Ruifeng Wang
> > > <Ruifeng.Wang at arm.com>; vcchunga at amazon.com; Dharmik Thakkar
> > > <Dharmik.Thakkar at arm.com>; hemant.agrawal at nxp.com; Ajit Khaparde
> > > (ajit.khaparde at broadcom.com) <ajit.khaparde at broadcom.com>;
> > > ferruh.yigit at intel.com; aboyer at pensando.io; lironh at marvell.com;
> > > dev at dpdk.org; nd <nd at arm.com>; Honnappa Nagarahalli
> > > <Honnappa.Nagarahalli at arm.com>; nd <nd at arm.com>
> > > Subject: RE: [PATCH v16 1/3] build: disable/enable drivers in Arm
> > > builds
> > >
> > > <snip>
> > >
> > > > > <removed parts which I think are not that relevant>
> > > > >
> > > > > > > > > > > > > The blocklist is, I think, agreed upon by everyone.
> > > > > > > > > > > > > The question is whether we want to support the
> > > > > > > > > > > > > allowlist alongside it and there seem to be
> > > > > > > > > > > > > enough reasons to do
> > > > that.
> > > > > > > > > > > > Sorry, may be this is answered already, but, what
> > > > > > > > > > > > additional benefit does allowlist provide over the blocklist?
> > > > > > > > > > > >
> > > > > > > > > > >
> > > > > > > > > > > VPP could use it:
> > > > > > > > > > > https://gerrit.fd.io/r/gitweb?p=vpp.git;a=blob;f=bui
> > > > > > > > > > > ld/e
> > > > > > > > > > > xt
> > > > > > > > > > > erna
> > > > > > > > > > > l/pa
> > > > > > > > > > > ckag es/dpdk
> > > > > > > > > > > .mk;h=c35ac84c27b19411a0cfdf9a3524fdf36024762c;hb=HE
> > > > > > > > > > > AD
> > > > > > > > > > >
> > > > > > > > > > > They're disabling almost everything so an allowlist would fit
> there.
> > > > > > > > > > > And they won't need to update the list when a new
> > > > > > > > > > > driver is added (which they won't need).
> > > > > > > > > > This is different from how we started this discussion.
> > > > > > > > > > The current discussion was for DPDK internal use. But
> > > > > > > > > > the one you are referencing above is for users of
> > > > > > > > > > DPDK. I am fine for providing the allow list for the
> > > > > > > > > > users of DPDK. But for DPDK internal, I think block
> > > > > > > > > > list is
> > > > > > > > enough.
> > > > > > > > > >
> > > > > > > > >
> > > > > > > > > That's an interesting suggestion. Jerin, what do you think?
> > > > > > > > > Why did you
> > > > > > > > want to have an allowlist? Would this work?
> > > > > > > >
> > > > > > > > # The very reason why VPP chooses to have allow list so
> > > > > > > > that they can control what needs to include.
> > > > > > > > # Another use case is like, in SoCs have fixed internal
> > > > > > > > devices, we can have optimized build for that can have
> > > > > > > > only allow list of the drivers that care about # For
> > > > > > > > server market, block list makes sense # For embedded SoC, allow
> list makes sense.
> > > > > > > For the embedded SoC, IMO, the upstream project could allow
> > > > > > > the compilation
> > > > > > for wider set of PMDs/libs. May be the end customer can use
> > > > > > the allow list to compile/use what is required?
> > > > > >
> > > > > > Just to understand, how end customer can enable allow list, if
> > > > > > DPDK build system does not support it?
> > > > > > Also to understand, If we are supporting blocklist, why not
> > > > > > have allowlist (I mean, both of them) as both are required as
> > > > > > it caters different use case as mention above. We can not
> > > > > > emulate allowlist with blocklist as each version of DPDK will
> > > > > > have new libraries and PMD's
> > > > which end user has no clue. Right?
> > > > >
> > > > > > I think, that is the reason why VPP is doing the allow list.
> > > > >
> > > > > I'm not sure what you mean by this, but to clarify, VPP likely
> > > > > would be using
> > > > the allowlist in this fashion, but that is not an arm specific
> > > > usecase. I think what Honnappa wanted to see was how the allowlist
> > > > could be used in an arm usecase (such as using it in an SoC configuration).
> > > >
> > > > There is nothing arm-specific here. Right? allowlist will be
> > > > common and will be used by all architecture. Right?
> > > Nothing Arm specific. I think for generic Arm servers platforms we
> > > can make sure that we allow for compilation of all the DPDK code. We
> > > can go ahead with implementing the allow list.
> > >
> >
> > I tried to actually use an allowlist and there are some problems with building
> apps and tests. When I tried to enable a random driver, such as
> common/sfc_efx, I've ran into dependency issues with apps:
> > app/meson.build:53:3: ERROR:  Tried to get unknown variable
> "static_rte_bus_vdev".
> >
> > This is because bus/vdev is not enabled (the allowlist only enabled
> net/sfc_efx). When I implemented dependency discovery in apps similar to
> which exists for drivers/libs, I was then unable to build tests (which is a special
> case app):
> > app/test/meson.build:427:1: ERROR:  Tried to get unknown variable
> "static_rte_bus_pci".
> >
> > This is becasue bus/pci is not enabled. If I understand the code correctly, the
> test dependencies are not matched to each test, meaning we can't disable the
> tests for which we don't have dependencies - we can only disable all tests.
> >
> > In general, this problem also affects the blocklist, i.e. meson build
> > -Ddisable_drivers=bus/pci produces
> > drivers/net/hinic/base/meson.build:34:0: ERROR:  Unknown variable
> "static_rte_bus_pci".
> > For blocklists, this is seldom going to be a problem, since most users won't
> disable "core" drivers.
> >
> > I'm not sure what is the best course of action. We can implement the allowlist
> and then we'll leave it up to implementers to implement allowlists that produce
> a working build. We could then optionally address the dependency issues
> brought by disabled drivers in a separate patch.
> >
> > Maybe we can just have a list of "core" drivers that can never be disabled
> (either using blocklists or allowlists)?
> >
> > Bruce, what do you think?
> >
> Hi,
> 
> from my experience this mainly seems to affect the bus drivers, specifically pci
> and vdev buses, which seem to be assumed to be always enabled. I agree it's
> probably not a real problem right now, though it would be nice for some testing
> purposes to have a build possible with "disable_drivers=*/*". For a solution I am
> happy for us to have whichever is easiest to implement - either refusing disabling
> of bus/pci and bus/vdev, or fixing the tests and apps to allow them to be built
> with reduced functionality.
> 
> Apart from those two drivers, I would hope that disabling everything else works.
> If not, we should definitely fix it.

I tried with just those two enabled and it works. I'll introduce a list of drivers that won't ever be disabled (ther other solution is more complicated and I think out of scope of this patch) and submit a new version.

> 
> /Bruce