[dpdk-dev] [PATCH 0/4] Add DSO symbol versioning to support backwards compatibility

[Neil Horman]

On Fri, Sep 19, 2014 at 07:18:36AM -0700, Venkatesan, Venky wrote:
> On 9/18/2014 12:14 PM, Neil Horman wrote:
> >On Thu, Sep 18, 2014 at 08:23:36PM +0200, Thomas Monjalon wrote:
> >>Hi Neil,
> >>
> >>2014-09-15 15:23, Neil Horman:
> >>>The DPDK ABI develops and changes quickly, which makes it difficult for
> >>>applications to keep up with the latest version of the library, especially when
> >>>it (the DPDK) is built as a set of shared objects, as applications may be built
> >>>against an older version of the library.
> >>>
> >>>To mitigate this, this patch series introduces support for library and symbol
> >>>versioning when the DPDK is built as a DSO.  Specifically, it does 4 things:
> >>>
> >>>1) Adds initial support for library versioning.  Each library now has a version
> >>>map that explicitly calls out what symbols are exported to using applications,
> >>>and assigns version(s) to them
> >>>
> >>>2) Adds support macros so that when libraries create incompatible ABI's,
> >>>multiple versions may be supported so that applications linked against older
> >>>DPDK releases can continue to function
> >>>
> >>>3) Adds library soname versioning suffixes so that when ABI's must be broken in
> >>>a fashion that requires a rebuild of older applications, they will break at load
> >>>time, rather than cause unexpected issues at run time.
> >>>
> >>>4) Adds documentation for ABI policy, and provides space to document deprecated
> >>>ABI versions, so that applications might be warned of impending changes.
> >>>
> >>>With these elements in place the DPDK has some support to allow for the extended
> >>>maintenence of older API's while still allowing the freedom to develop new and
> >>>improved API's.
> >>>
> >>>Implementing this feature will require some additional effort on the part of
> >>>developers and reviewers.  When reviewing patches, must be checked against
> >>>existing exports to ensure that the function prototypes are not changing.  If
> >>>they are, the versioning macros must be used, and the library export map should
> >>>be updated to reflect the new version of the function.
> >>>
> >>>When data structures change, if those structures are application accessible,
> >>>apis that accept or return instances of those data structures should have new
> >>>versions created so that users of the old data structure version might co-exist
> >>>at the same time.
> >>Thanks for your efforts.
> >>But I feel this change has too many constraints for the current status of
> >>the DPDK. It's probably too early to adopt such policy.
> >>
> >I think you may be misunderstanding something.  What constraints do you beleive
> >that this patch imposes?  Note it doesn't in any way prevent changes to the ABI
> >of the DPDK, but rather gives us infrastructure to support multiple ABI
> >revisions at the same time, so that applications built against DPDK shared
> >libraries can continue to function properly at least for some time until we
> >decide to deprecate that ABI level.
> >
> >This is all based on the versioning strategy outlined here:
> >http://www.akkadia.org/drepper/dsohowto.pdf
> >
> >That may help clarify things for you.
> >
> >>By the way, this versioning doesn't cover structure changes.
> >No, it doesn't.  No link-time mechanism does so.
> >
> >>How could it be managed?
> >Thats a subject that is open to discussion, but my initial thinking is that we
> >need to handle it on a case by case basis:
> >
> >* For minor updates, where allocation of a structure is done on the heap and new
> >fields need to be added, appending them to the end of a structure and providing
> >an initial value is sufficient.
> >
> >* For major changes, where fields need to be removed, or re-arranged, mostly
> >likely the API surfaces which accept or return those structures as
> >inputs/outputs will need to have new versions written to accept the new version
> >of the structure, and internally we will have to support both formats for a time
> >(according to the policy I documented, that is currently a single major
> >release).  I.e. if you want to change struct foo, which is accepted as a
> >parameter for the function bar(struct foo *ptr), then for a release we would
> >need to create struct foo_v2 with the new format, map a new function foo_v2 to
> >the exported foo@@DPDK_1.(X+1), and internally make the foo functions understand
> >both the origional and v2 versions of the structure.  Then in DPDK release
> >1.X+2, we can remove the old version after posting a deprecation notice with
> >version 1.(X+1)
> >
> >>Don't you think it would be more reliable if managed by packaging?
> >Solving this with packaging defeats the purpose of having shared libraries at
> >all.  While packaging each version of the dpdk separately is possible stopgap
> >solution, in that it allows applications to link to differing versions of the
> >library independently, but that negates any expectation of timely bugfixes for
> >any given version of the DPDK.  That is to say, if you package things this way,
> >and wind up with several parallel versions of the same package, and for any
> >bugfix that comes out upstream, the packager then has the responsibility to
> >adapt that fix to each package.  Thats an unscalable solution, and not something
> >any packager is going to undertake willingly.  I did a hybrid version of this in
> >fedora for exactly that reason.  I packaged the dpdk into its own directory, but
> >have every intention of changing that directory every major release, so that
> >application writers can clearly see when they need to stop updating the dpdk,
> >lest their applications stop linking. I'm not going to have multiple dpdk
> >packages to maintain in parallel, thats just too much work.
>  I do think that this is something that needs to be addressed in the DPDK
> (and not with packaging). Besides what Neil points out, DPDK can work with a
> lot of linux distros, and other operating systems too. Replicating the work
> with each (even if it is just two or three that we focus on) is wasteful.

While its nice and generous of the upstream community to provide packaging
samples, its also not something that that upstream development should really
need to worry about.  Packaging is really meant to address the needs of the
distribution doing the packaging.  Relying on it to solve versioning problems
in place of a more appropriate solution just leads to fragmentation accross
distributions, as invariably different distros will manage that versioning
differently, which leads to applications needing to manage versioning
differently, which is what I'm trying to avoid :)

> >>Thank you for opening this discussion with a constructive proposal.
> >>Let's check it later on once structures will be more stable since
> >>performance is the most critical target.
> Performance will always be a critical target for us. However, as we find
> more problems that need to be solved, we will add new libraries and new
> APIs. That can't be a reason to
> >If I'm being honest, I have to say thats a cop out answer.  We all know that
> >structure stability isn't a priority for the DPDK, nor will it ever be in all
> >likelyhood.  It will continue to evolve and grow as the hardware does.  And this
> >patch set doesn't prevent that from happening.  All it does is provide some
> >level of stability in the API for a period of time to let 3rd party application
> >writers write and package applications with some allowance of time to keep up
> >with upstream changes on their own schedule.
> >
> >I grant you that writing a good API for a shared library is difficult, but
> >(and feel free to disagree with this), if we don't start enforcing policies that
> >require good API design, its not going to happen on its own.  This patch set
> >will highlight those API points which are difficult to maintain accross major
> >releases, and force us to address and improve them.  To that end I've already
> >begun talking to some of the individual library maintainers off list to address
> >some of the API aspects that I have concerns about (exporting variable rather
> >than accessor functions, structures that don't need to be visible to users,
> >etc), and they've started reviewing them.  We can make this better, but we can't
> >just say later, because theres no roadmap that lists structure stability as a
> >line item.  As hardware improves, structures will change to operate more
> >efficiently or support more features.  Without a hard plan, the initial goals of
> >the DPDK (high performance networking) will relegate ABI to such a low priority
> >that it will never be addressed.
> Neil, you're spot on here. To an extent, there will always be changes to the
> API for various reasons. We've done a reasonable job of managing changes so
> far, but there are going to be changes. I do think that this patch provides
> a way for applications to manage through those changes at the pace they can
> absorb.
> 
Thank you.  Let me be clear for anyone who might not have heard me say this
before.  In no way am I trying to limit the evolution of the DPDK ABI or API.
All I'm trying to do here is provide some infrastructure that allows for exiting
ABI to carry on for a minimal guaranteed period of time (currently set to at
least one release beyond its deprecation notification).  I won't lie, that does
mean that API design and maintenence will be a potential extra work effort, but
I don't think thats a bad thing, as doing so will lead to better API design
(that will hopefully just last longer naturally), and it will help expand the
reach of the DPDK as application writers can better separate their development
cycles from that of the DPDK itself.

> Secondly, one other usage scenario that we will run into is when apps using
> different versions of DPDK are installed on the same system - this patch at
> least gives us a start point to at least flag this problem.
Yeah, I'm not sure how I'll deal with that from a packaging standpoint yet, but
soname versioning at least gives me a tool to make it easier to do using
whatever method I choose.

> >
> >To that end, can we discuss specifics?  Can you ennumerate direct points that
> >you feel make this patch unworkable at this time?  I know you mentioned some
> >above, and I think I addressed them (though please ask follow up questions if
> >I've been unclear).  What other concerns do you have?
> >
> >Neil
>  This is a good start - I've put it into my development systems and will let
> you know if I find anything that is a showstopper.
> 

Thanks!
Neil

> Regards,
> -Venky
>