[dpdk-dev] Further fun with ABI tracking

[Christian Ehrhardt]

On Thu, Feb 23, 2017 at 7:48 PM, Ferruh Yigit <ferruh.yigit at intel.com>
wrote:

> Can you please describe this option more?
>

Of course, I happy about any engagement/discussion to have on that.
Much better than silently staying in queue.


> Does is mean for each DPDK release, distro will release all libraries?
>

First of all it is opt-in. If nobody does change the default setting (="")
to
anything nothing happens.

A distribution _CAN_ use the feature to reliably avoid collisions between
DPDK releases and allow concurrent installations more easily.


> For 16.07:
> acl.2, eal.2, ethdev.4, pdump.1
>
> For 16.11:
> acl.2, eal.3, ethdev.5, pdump.1
>

That example is what we did so far, trying to follow the DPDK ABI
versioning.
But that caused the issue I described with (new)pdump.1->eal.3 and the base
app->eal2


Will dpdk package have following packages:
> acl.16.07.2, eal.16.07.2, ethdev.16.07.4, pdump.16.07.1
> acl.16.11.2, eal.16.11.3, ethdev.16.11.5, pdump.16.11.1
>

I thought on that, but Jan correctly brought up that if we do override we
should
as well trivialize and override fully - ignoring the per lib LIBABIVER.
So it will not be eal.16.11.3 but instead just eal.16.11   (no subversion,
as there
is no need). If that is wanted that can easily be done, please let me know
if I
should run a v2 with that.

And for initial OVS usecase, will it be:
>
> OVS
>  +---> eal.16.07.2
>  +---> pdump.16.11.1
>         +---> eal.16.11.3
>
>
Not quite, the usecase would look like that:
The current DPDK generates LIBABIVER versions: eal.3, pdump.1, ...
OVS
 +---> eal.3
 +---> pdump.1
        +---> eal.3

Note: Packages are initially carried forward from the former Distrobution
release,
so the next release would start as the former has ended:
OVS
 +---> eal.3
 +---> pdump.1
        +---> eal.3

Then the new DPDK would come in and using this feature would then
generate all as major version: eal.17.02, pdump.17.02, ...
But since OVS was not recompiled yet AND there is no collision OVS
would still look like:
OVS
 +---> eal.3
 +---> pdump.1
        +---> eal.3

Then we can recompile OVS and it will become
OVS
 +---> eal.17.02
 +---> pdump.17.02
        +---> eal.17.02

Into the future with more apps depending on DPDK there can be many
scenarios:
1. all are fine rebuilding, there will be no dependency left to the older
dpdk
    and it will be autoremoved after all upgrades
2. some packages are slow to adapt, but that is fine we can still provide
the
    old dependencies at the same time if needed
3. the time in between #2 and #1 is not wreaking havok as the
cross-dependency
    issue is no more


> Assuming above understanding is correct J :
>
> - If same version of the library will be delivered for each DPDK
> release, what is the benefit of having fine grained libraries really?
>

The benefit of the fine grained versioning is for other types of
distributing DPDK.
Recognizing an ABI bump for lib-consuming developers, bundling it directly
with your app, ...



> - Above OVS usage still does not look right, I don't believe this is the
> intention when library level dependency resolving introduced.
>
> Overall I am for single library, but I can see the benefit of having
> multiple small libraries, that is why I vote for option 4 in your
> initial mail.
>

Single library would solve it as well, but as mentioned and you all
remember there
were people with reasons for it that I could not challenge being too far
out of the
application scenarios they had in mind.

And I agree this can cause problem if not automated, but we already know
> the library dependencies, I think a script can be developed to warn a
> least, and they can be updated manually.
>
> And isn't the purpose of increasing LIBABIVER to notify application that
> library is modified and can't be used with that app anymore.
> For DPDK, even if the library is not changed, if another library that it
> depends modified, this may mean the behavior of the library may be
> changed, so it makes sense to me if library notifies the user for this
> case, by increasing its version.
>
> Yes this makes effect of increasing a core library version big, but I
> believe this is also true, increasing a core library version almost
> means increasing dpdk version.
>

Interesting - thanks for sharing your opinion here - I rethought that for a
while now.

While this could work I consider it inferior to the approach I submitted
in the patch yesterday [1] for the following reasons:

- If we bump infecting (looking at the recent history) we most likely end up
  bumping all libraries at least every other release. Now there isn't much
  different in bumping all of them +1 or just using a single increasing
version.
  Except you could miss a few bumps or track it wrong.

- The new Feature is opt-in, allowing those who want to do that major bump;
  But at the same time allowing those who won't to keep on tracking each lib
  individually and build/deliver it that way.

- I learned (often the hard way) that to be different often causes problems
  that are hard to foresee.
  The infecting ABI would be "DPDK is different" again, while the major
  override is somewhat established.

For now I'd suggest taking the opt-in feature as suggested in [1] as a means
for those who need it (like us and maybe more downstreams over time).
If DPDK is evolving to become more stable and develops a feature like
the #4 "infecting-abi-bump + tracking" it can still be picked us later and
by anybody else who wants needs it.
It will then "just" be dropping a config option we set before to get back.


TL;DR: I think DPDK is not stable enough to make option #4 worth
implementing for now to make a difference worth (but would cause
lot of work and error potential). But since my code [1] implementing
approach #1 and a later approach #4 in the future are not mutually
exclusive I'd ask to go for #1 now and #4 later if one needs and
implements it.

[1]: http://dpdk.org/ml/archives/dev/2017-February/058121.html


-- 
Christian Ehrhardt
Software Engineer, Ubuntu Server
Canonical Ltd