[dpdk-dev] Further fun with ABI tracking

Christian Ehrhardt christian.ehrhardt at canonical.com
Tue Feb 14 11:52:00 CET 2017


Hi,
when moving to DPDK 16.11 Debian/Ubuntu packaging of DPDK has hit a new
twist on the (it seems reoccurring) topic of DPDK ABI tracking.

I have found, ... well I don't want to call it solution ..., let's say a
crutch to get around it for the moment. But I wanted to use the example I
had to share a few thoughts on it and to kick off a wider discussion.


*## In library cross-dependencies plus partial ABI bumps ##*

Since the day moving away from the combined shared library we had several
improvements on tracking the ABI versions. These days [1] we have LIBABIVER
per library and it gets bumped to reflect it is breaking with former
versions e.g. removing symbols.

Now in the 16.11 release the ABIs for cryptodev, eal and ethdev got bumped
by [2] and [3].

OTOH please remember that in general two versions of a shared library in
the usual sense are meant to be able to stay alongside on a system without
hurting each other. I picked a random one on my system.
Package              Library
libisc-export160: /lib/x86_64-linux-gnu/libisc-export.so.160
libisc-export160: /lib/x86_64-linux-gnu/libisc-export.so.160.0.0
libisc-export95: /lib/x86_64-linux-gnu/libisc-export.so.95
libisc-export95: /lib/x86_64-linux-gnu/libisc-export.so.95.5.0
Some link against the new, some against the old library - all fine.
Usually most programs can just be rebuilt against the new library and after
some time the old one can be dropped. That mechanism gives downstream
distributions a way to handle transitions and consumers of libraries which
might not all be ready for the same version every time.
And since the per lib versioning with LIBABIVER and and the version maps we
are good - in fact we qualify for all common cases on [4].

Now in DPDK of those libraries that got an ABI bump eal and ethdev are part
of those which most of us consider "core libraries" and most other libs and
pmds link to them.
And here DPDK continues to be special, due to that inter-dependency with
old and new libraries installed on the same system the following happens on
openvswitch built for an older version of dpdk:
ovs-vswitchd-dpdk
    librte_eal.so.2 => /usr/lib/x86_64-linux-gnu/librte_eal.so.2
    librte_pdump.so.1 => /usr/lib/x86_64-linux-gnu/librte_pdump.so.1
        librte_eal.so.3 => /usr/lib/x86_64-linux-gnu/librte_eal.so.3

You can see that Openvswitch itself depends on the "old" librte_eal.so.2.
But because  librte_pdump.so.1 did not get an ABI bump it got upgraded to
the newer version from DPDK 16.11.
But since the "new" pdump got built with the new DPDK 16.11 it depends on
the "new" librte_eal.so.3.
And having both in the same executable space at the same time causes
segfaults and pain.

As I said for now I have passed the issue with a crutch that I'm not proud
of and I'd like to avoid in the future. For that I'm reaching out to you
with several suggestions to discuss.


*## Thoughts ##*
None of these seems like a perfect solution to me yet, but clearly good to
start discussions on them.

Options that were in discussion so far and that we might adopt next cycle
(some of these are upstream changes, some downstream, some require both to
change - but any of them should have an ack upstream so that we are
agreeing how to proceed with those cases).

1. Downstreams to insert Major version into soname
Distributions could insert the DPDK major version (like 16.11) into the
soname and package names. A common example of this is libboost [5].
That would perfectly allow 16.07.<LIBABIVER> to coexist with
16.11.<LIBABIVER> even if for a given library LIBABIVER did not change.
Yet it would mean that anything depending on the old library will have to
be recompiled to pick up the new code, even if it depends on an ABI that is
still present in the new release.
Also - not a technical reason - but it is clearly more work to force update
all dependencies and clean out old packages for every release.


2. ABI Ranges
One could argue that due to the detailed tracking of functions DPDK is
already close to track not ABI levels but actually ABI ranges. DPDK could
track LIBABIVERMIN and LIBABIVER.
Every time functionality is added LIBABIVER would get bumped, but
LIBABIVERMIN only gets moved to the OLDEST still supported ABI when things
are dropped.
So on a given library librte_foo you could have LIBABIVER=5 and
LIBABIVERMIN=3. The make install would then install the shared lib as:
librte_foo.so.5
and additionally links for all compatible versions:
librte_foo.so.3 -> librte_foo.so.5
librte_foo.so.4 -> librte_foo.so.5
Yet, while is has some nice attributes this might make DPDK even more
special and cause ABI level proliferation over time.
Also even with this in place, changes moving LIBABIVERMIN "too fast" (too
fast is different for each downstream) could still cause an issue like the
one I initially described.


3. A lot of conflicts
In packaging one can declare a package to conflict with another package [6].
Now we could declare e.g. librte_eal3 to conflict with librte_eal2 (and the
same for all other bumps).
That would make them not coinstallable, and working on a new release would
mean that all former consumers would become not installable as well and
have to be rebuilt before they all could migrate [7] together.
That "works" in some sense, but it denies the whole purpose of versioned
library packages (to be coninstallable, to allow different library
consumers to depent on different versions)


4. ABI bump is infecting
Another way might be to also bump any dependent DPDK library.
So when core libs like eal are ABI bumped likely all libs would get a bump.
If only e.g. mempool gets a bump only those other parts using it would be
bumped as well.
To some extend this might still proliferate ABI versions more than one
would like.
Also it surely is hard to track if not automated - think of dependencies
that are existing only in certain config cases.

5. back to single ABI
For the sake of giving everybody a chance to re-open old wounds I wanted to
mention that DPDK could also decide to go back to a single ABI again.
This could (but doesn't have to!) be combined with having a single .so file
again.
To decide for this might be a much cleaner and easier to track way to #4.

6. More
I'm sure there are more approaches to this, feel free to come up with more.

I'm sure my five suggestions alone will make the thread messy, Maybe we do
this in two rounds, sorting out the insane and identifying the preferred
ones to then in a second run focus on discussing and maybe implementing the
details of what we like.


[1]: http://dpdk.org/browse/dpdk/tree/doc/guides/contributing/versioning.rst
[2]: http://dpdk.org/browse/dpdk/commit/?id=d7e61ad3ae36
[3]: http://dpdk.org/browse/dpdk/commit/?id=6ba1affa54108
[4]: https://wiki.debian.org/TransitionBestPractices
[5]: https://packages.debian.org/sid/libboost1.62-dev
[6]:
https://www.debian.org/doc/debian-policy/ch-relationships.html#s-conflicts
[7]: https://wiki.ubuntu.com/ProposedMigration

P.S. I beg a pardon for the wall of text

-- 
Christian Ehrhardt
Software Engineer, Ubuntu Server
Canonical Ltd


More information about the dev mailing list