[dpdk-dev] [EXT] [RFC PATCH 0/2] introduce __rte_internal tag

Neil Horman nhorman at tuxdriver.com
Thu Jun 6 17:03:54 CEST 2019


On Thu, Jun 06, 2019 at 02:02:03PM +0000, Jerin Jacob Kollanukkaran wrote:
> > -----Original Message-----
> > From: Neil Horman <nhorman at tuxdriver.com>
> > Sent: Thursday, June 6, 2019 7:05 PM
> > To: Jerin Jacob Kollanukkaran <jerinj at marvell.com>
> > Cc: Bruce Richardson <bruce.richardson at intel.com>; dev at dpdk.org;
> > Thomas Monjalon <thomas at monjalon.net>
> > Subject: Re: [EXT] [RFC PATCH 0/2] introduce __rte_internal tag
> > 
> > On Thu, Jun 06, 2019 at 12:04:57PM +0000, Jerin Jacob Kollanukkaran wrote:
> > > > -----Original Message-----
> > > > From: Neil Horman <nhorman at tuxdriver.com>
> > > > Sent: Thursday, June 6, 2019 5:04 PM
> > > > To: Jerin Jacob Kollanukkaran <jerinj at marvell.com>
> > > > Cc: Bruce Richardson <bruce.richardson at intel.com>; dev at dpdk.org;
> > > > Thomas Monjalon <thomas at monjalon.net>
> > > > Subject: Re: [EXT] [RFC PATCH 0/2] introduce __rte_internal tag
> > > >
> > > > On Thu, Jun 06, 2019 at 09:44:52AM +0000, Jerin Jacob Kollanukkaran
> > wrote:
> > > > > > -----Original Message-----
> > > > > > From: Neil Horman <nhorman at tuxdriver.com>
> > > > > > Sent: Wednesday, June 5, 2019 11:41 PM
> > > > > > To: Bruce Richardson <bruce.richardson at intel.com>
> > > > > > Cc: Jerin Jacob Kollanukkaran <jerinj at marvell.com>;
> > > > > > dev at dpdk.org; Thomas Monjalon <thomas at monjalon.net>
> > > > > > Subject: Re: [EXT] [RFC PATCH 0/2] introduce __rte_internal tag
> > > > > >
> > > > > > On Wed, Jun 05, 2019 at 05:45:41PM +0100, Bruce Richardson wrote:
> > > > > > > On Wed, Jun 05, 2019 at 04:24:09PM +0000, Jerin Jacob
> > > > > > > Kollanukkaran
> > > > > > wrote:
> > > > > > > > > -----Original Message-----
> > > > > > > > > From: Neil Horman <nhorman at tuxdriver.com>
> > > > > > > > > Sent: Sunday, May 26, 2019 12:14 AM
> > > > > > > > > To: dev at dpdk.org
> > > > > > > > > Cc: Neil Horman <nhorman at tuxdriver.com>; Jerin Jacob
> > > > > > > > > Kollanukkaran <jerinj at marvell.com>; Bruce Richardson
> > > > > > > > > <bruce.richardson at intel.com>; Thomas Monjalon
> > > > > > > > > <thomas at monjalon.net>
> > > > > > > > > Subject: [EXT] [RFC PATCH 0/2] introduce __rte_internal
> > > > > > > > > tag
> > > > > > > > >
> > > > > > > > > Hey-
> > > > > > > > > 	Based on our recent conversations regarding the use of
> > > > > > > > > symbols only meant for internal dpdk consumption (between
> > > > > > > > > dpdk libraries), this is an idea that I've come up with
> > > > > > > > > that I'd like to get some feedback on
> > > > > > > > >
> > > > > > > > > Summary:
> > > > > > > > > 1) We have symbols in the DPDK that are meant to be used
> > > > > > > > > between DPDK libraries, but not by applications linking to
> > > > > > > > > them
> > > > > > > > > 2) We would like to document those symbols in the code, so
> > > > > > > > > as to note them clearly as for being meant for internal
> > > > > > > > > use only
> > > > > > > > > 3) Linker symbol visibility is a very coarse grained tool,
> > > > > > > > > and so there is no good way in a single library to mark
> > > > > > > > > items as being meant for use only by other DPDK libraries,
> > > > > > > > > at least not without some extensive runtime checking
> > > > > > > > >
> > > > > > > > >
> > > > > > > > > Proposal:
> > > > > > > > > I'm proposing that we introduce the __rte_internal tag.
> > > > > > > > > From a coding standpoint it works a great deal like the
> > > > > > > > > __rte_experimental tag in that it expempts the tagged
> > > > > > > > > symbol from ABI constraints (as the only users should be
> > > > > > > > > represented in the DPDK build environment).  Additionally,
> > > > > > > > > the __rte_internal macro resolves differently based on the
> > > > > > > > > definition of the BUILDING_RTE_SDK flag (working under the
> > > > > > > > > assumption that said flag should only ever be set if we
> > > > > > > > > are actually building DPDK libraries which will make use
> > > > > > > > > of internal calls).  If the BUILDING_RTE_SDK flag is set
> > > > > > > > > __rte_internal resolves to __attribute__((section
> > > > > > > > > "text.internal)), placing it in a special text section
> > > > > > > > > which is then used to validate that the the symbol appears
> > > > > > > > > in the INTERNAL section of the corresponding library version
> > map).
> > > > > > > > > If BUILDING_RTE_SDK is not set, then __rte_internal
> > > > > > > > > resolves to
> > > > > > __attribute__((error("..."))), which causes any caller of the
> > > > > > tagged function to throw an error at compile time, indicating
> > > > > > that the symbol is not available for external use.
> > > > > > > > >
> > > > > > > > > This isn't a perfect solution, as applications can still
> > > > > > > > > hack around it of course,
> > > > > > > >
> > > > > > > > I think, one way to, avoid, hack around could be to,
> > > > > > > >
> > > > > > > > 1) at config stage, create  a random number for the build
> > > > > > > > 2) introduce RTE_CALL_INTERNAL macro for calling internal
> > > > > > > > function, compare the generated random number for allowing
> > > > > > > > the calls to make within the library. i.e leverage the fact
> > > > > > > > that external library would never know the random number
> > > > > > > > generated for the DPDK build
> > > > > > and internal driver code does.
> > > > > > > >
> > > > > > > Do we really need to care about this. If have some determined
> > > > > > > enough to hack around our limitations, then they surely know
> > > > > > > that they have an unsupported configuration. We just need to
> > > > > > > protect against inadvertent use of internals, IMHO.
> > > > > > >
> > > > > > I agree, I too had thought about doing some sort of internal
> > > > > > runtime checking to match internal only symbols, such that they
> > > > > > were only accessable by internally approved users, but it
> > > > > > started to feel like a great
> > > > deal of overhead.
> > > > > > Its a good idea for a general mechanism I think, but I believe
> > > > > > the value here is more to internally document which apis we want
> > > > > > to mark as being for internal use only, and create a lightweight
> > > > > > roadblock at build time to catch users inadvertently using them.
> > > > > > Determined users will get around anything, and theres not much
> > > > > > we can do to stop
> > > > them.
> > > > >
> > > > > I agree too. IMHO, Simply having following items would be enough
> > > > >
> > > > > 1) Avoid exposing the internal function prototype through public
> > > > > header files
> > > > > 2) Add @internal to API documentation
> > > > > 3) Just decide the name space for internal API for tooling(i.e not
> > > > > start with rte_ or so) Using objdump scheme to detect internal
> > > > > functions
> > > > requires the the library to build prior to run the checkpatch.
> > > > >
> > > >
> > > > No, I'm not comfortable with that approach, and I've stated why:
> > > > 1) Not exposing the functions via header files is a fine start
> > > >
> > > > 2) Adding internal documentation is also fine, but does nothing to
> > > > correlate the code implementing those functions to the
> > > > documentation.  Its valuable to have a tag on a function identifying it as
> > internal only.
> > > >
> > > > 3) Using naming conventions to separate internal only from
> > > > non-internal functions is a vague approach, requiring future
> > > > developers to be cogniscent of the convention and make the
> > > > appropriate naming choices.  It also implicitly restricts the
> > > > abliity for future developers to make naming changes in conflict
> > > > with that convention
> > >
> > > Enforcing the naming convention can be achieved through tooling as well.
> > >
> > Sure, but why enforce any function naming at all, when you don't have to.
> 
> May I ask,  why to  enforce __rte_internal, when you don't have to
> 

Because its more clear.  Implicitly deciding that any function not prefixed with
rte_ is internal only does nothing to prevent a developer from accidentally
naming a function incorrectly, exporting it, and allowing a user to call it. We
can move headers all you want, but we provide an ABI guarantee to end users, and
developers should have a way to clearly record that without having to check the
documentation for each function that an application developer wants to use.

The long and the short of it for me is that I want a way for developers to opt
their code into an internal only condition, not to just document it as such
and hope its up to date.  If they tag a function as __rte_internal then its
clearly marked as internal only, they have checks to ensure that its in the
INTERNAL section of the version map, and should that header somehow get
externally exported (see rte_mempool_check_cookies for an example of how thats
happened), users are prevented from using them at build time, rather than having
to ask questions on the list, or read documentation after an error to find out
"oops, shouldn't have done that".

I think you'll find that going through all the header files, and bifurcating
them to public and private headers is a much larger undertaking than just
tagging those functions accordingly.  a quick scan of all our header file for
the @internal tag shows about 260 instances of such functions, almost all of
which are published to applications.  All of those functions would have to be
moved to private headers, and their requisite C files would need to be updated
to include the new header.  with the use of __rte_internal, we just have tag the
functions as such, which can be handled with a cocinelle or awk script.

Neil
 

> > 
> > > >
> > > > 4) Adding a tag like __rte_internal creates an interlock whereby,
> > > > not only are internal functions excused from ABI constraints, but
> > > > forces developers to intentionally mark their internal functions as
> > > > being internal in the code, which is beneficial to clarlity of understanding
> > during the development process.
> > >
> > > No issues in adding __rte_internal. But, I am against current
> > > implementaion, Ie. adding objdump dependency
> > That dependency already exists for the __rte_external flag
> 
> Sorry, I could not see the dependency.
> 
> [master][dpdk.org] $ grep -ri "objdump" devtools/
> [master][dpdk.org] $ grep -ri "objdump" usertools/
> [master][dpdk.org] $ grep -ri "__rte_external" *
> 
> > 
> > > to checkpatch i.e developer has to build the library first so  that
> > > checkpatch can can know, Is it belongs to internal section or not?
> > >
> > What developer is running checkpatch/posting patches without first building
> > their changes?
> 
> # it is not developer, The CI/CD tools can quicky check the sanity of patches
> before the build itself. Why to add unnecessary dependency?
> # If some PMD is not building if the requirements are not meet(say AES NI PMD for crypto)
> then how do take care of the dependency.
> 
> 
> > 
> > 
> > > >
> > > > 5) Adding a tag like __rte_internal is explicit, and allows
> > > > developers to use a single header file instead of multiple header
> > > > files if they so choose
> > > >
> > > > We went through this with experimental symbols as well[1], and it
> > > > just makes more sense to me to clearly document in the code what
> > > > constitutes an internal symbol rather than relying on naming
> > > > conventions and hoping that developers read the documentation before
> > > > exporting a symbol publically.
> > > >
> > > >
> > > > [1] https://mails.dpdk.org/archives/dev/2017-December/083828.html
> > > > > >
> > > > > > If we really wanted to go down that road, we could use a
> > > > > > mechainsm simmilar to the EXPORT_SYMBOL / EXPORT_SYMBOL_GPL
> > > > > > infrastructure that the kernel uses, but that would required
> > > > > > building our own custom linker script, which seems like overkill here.
> > > > > >
> > > > > > Best
> > > > > > Neil
> > > > > >
> > > > > > > /Bruce
> > > > > > >
> > > > >
> > >
> 


More information about the dev mailing list