[dpdk-dev] versioning and maintenance

[Thomas Monjalon]

Hi,

2014-11-19 10:13, Neil Horman:
> On Wed, Nov 19, 2014 at 11:35:08AM +0000, Bruce Richardson wrote:
> > On Wed, Nov 19, 2014 at 12:22:14PM +0100, Thomas Monjalon wrote:
> > > Following the discussion we had with Neil during the conference call,
> > > I suggest this plan, starting with the next release (2.0):
> > > 	- add version numbers to libraries
> > > 	- add version numbers to functions inside .map files
> > > 	- create a git tree dedicated to maintenance and API compatibility
> > > 
> > > It means these version numbers must be incremented when breaking the API.
> > > Though the old code paths will be maintained and tested separately by volunteers.
> > > A mailing list for maintenance purpose could be created if needed.
> > >
> > Hi Thomas,
> > 
> > I really think that the versionning is best handled inside the main repository
> > itself. Given that the proposed deprecation policy is over two releases i.e. an
> > API is marked deprecated in release X and then removed in X+1, I don't see the
> > maintaining of old code paths to be particularly onerous.
> > 
> > /Bruce
> 
> I agree with Bruce, even if it is on occasion an added workload, its not the
> sort of thing that can or should be placed on an alternate developer.  Backwards
> compatibility is the sort of thing that has to be on the mind of the developer
> when modifying an API, and on the mind of the reviewer when reviewing code.  To
> shunt that responsibility elsewhere invites the opportunity for backwards
> compatibilty to be a second class citizen who's goal will never be reached,
> because developers instituting ABI changes will never care about the
> consequences, and anyone worrying about backwards compatibility will always be
> playing catch up, possibly allowing ABI breaks to slip through.
> 
> Neil
 
Before taking a decision, we should detail every concern.

1/
Currently there are not a lot of API refactoring because DPDK is well tailored
for x86 and Intel NICs. But we are seeing that new CPU and new NICs to support
would require some adaptations.

2/
I'm curious to know how you would handle a big change like the recent mbuf rework.
Should we duplicate the structure and all the functions using mbuf?

3/
Should we add new fields at the end of its structure to avoid ABI breaking?

4/
Developers contribute because they need some changes. So when breaking
an API, their application is already ready for the new version.
I mean the author of such patch is probably not really motivated to keep ABI
compability and duplicate the code path.

5/
Intead of simply modifying an API function, it would appear as a whole new
function with some differences compared to the old one. Such change is really
not convenient to review.

6/
Testing ABI compatibility could be tricky. We would need a tool to check it's
mostly OK. The good place for such a tool is in app/test. It was designed to be
the unit tests of the API.

7/
This system would allow application developpers to upgrade DPDK to n+1 without
rebuilding. But when upgrading to n+2, they should have adapted their
application to comply with n+1 API (because n will be removed).
So this solution offers a delay between the upgrade decision and the
app work. Note that they could prepare their application before upgrading.
Anyway, an upgrade should be tested before doing it effectively. The behaviour
of the application could change and require some adaptations.

8/
How to handle a change in the implementation of a PMD which severely impact
the application? Example: an ol_flag was mis-used and the application has
a workaround to interpret this flag but it's now incompatible with the fix.

9/
When we don't want to adapt an application, it means the development is
finished and we don't care about the new features of the library.
So I wonder if it wouldn't be more appropriate to provide stable releases
with true maintenance to such users. I understood that is what Redhat provides
to their customers.

Hope this discussion will bring a clear idea of what should be done with
which implications.
Thanks
-- 
Thomas