[dpdk-dev] How to manage new APIs added after major ABI release?

[Ferruh Yigit]

On 12/11/2019 3:55 PM, Bruce Richardson wrote:
> On Wed, Dec 11, 2019 at 03:46:10PM +0000, Ferruh Yigit wrote:
>> On 12/11/2019 3:02 PM, Thomas Monjalon wrote:
>>> 11/12/2019 14:30, Ferruh Yigit:
>>>> On 12/11/2019 1:11 PM, Neil Horman wrote:
>>>>> On Tue, Dec 10, 2019 at 11:56:28AM +0000, Ferruh Yigit wrote:
>>>>>> Hi,
>>>>>>
>>>>>> With new process, the major ABI releases will be compatible until it is
>>>>>> deprecated (until next LTS for now),
>>>>>> like current ABI version is 20 in DPDK_19.11 and DPDK versions until DPDK_20.11
>>>>>> will be ABI compatible with this version.
>>>>>>
>>>>>> But if we introduce a new API after major ABI, say in 20.02 release, are we
>>>>>> allowed to break the ABI for that API before DPDK_20.11?
>>>>>>
>>>>>> If we allow it break, following problem will be observed:
>>>>>> Assume an application using .so.20.1 library, and using the new API introduced
>>>>>> in 20.02, lets say foo(),
>>>>>> but when application switches to .so.20.2 (released via DPDK_20.05), application
>>>>>> will fail because of ABI breakage in foo().
>>>>>>
>>>>>> I think it is fair that application expects forward compatibility in minor
>>>>>> versions of a shared library.
>>>>>> Like if application linked against .so.20.2, fair to expect .so.20.3, .so.20.4
>>>>>> etc will work fine. I think currently only .so.20.0 is fully forward compatible.
>>>>>>
>>>>>> If we all agree on this, we may need to tweak the process a little, but before
>>>>>> diving into implementation details, I would like to be sure we are in same page.
>>>>>>
>>>>> Yes, I agree with the assertion.  Once an ABI is fixed, it must be compatible
>>>>> with all future minor releases subsequent to the fixing of that ABI, until the
>>>>> next major update.  That is to say, once you release ABI_20, all minor updates
>>>>> 20.01, 20.02, etc must be compatible with ABI_20 until such time as ABI_21 is
>>>>> released.
>>>>
>>>> There is a slight difference. All minor versions already compatible with ABI_20,
>>>> like: 20.01, 20.02, 20.03 are ABI compatible with 20.0 (which defines ABI_20).
>>>>
>>>> Question is if 20.03 should be compatible with 20.02?
>>>>
>>>> This can happen if a new API is introduced in 20.2 and ABI has broken for that
>>>> API in 20.3, so an ABI compatibility issue created between 20.03 & 20.02,
>>>> meanwhile both are compatible with ABI_20.
>>>>
>>>> I can see two options:
>>>> a) New APIs are introduced only when we switch to new major ABI version. But if
>>>> we switch to longer (2 years) ABI compatibility, I think this is unacceptable to
>>>> wait up to two years to have (non experimental) APIs.
>>>
>>> I agree we should allow to add a new stable API in the middle of an ABI lifecycle.
>>>
>>>> b) APIs added in minor version will be part of ABI_20 after that point and same
>>>> rules will apply to them. Like if and API has introduced in 20.2, it is not
>>>> allowed to be broken until next major ABI version.
>>>
>>> Yes I think it is compliant with the agreed policy.
>>
>> So I think two minor changes are required in the process to reflect this,
>> 1) In ABI policy [1], it mentions in minor release both ABI_20 and ABI_21 can
>> exist together, also in graph it says for minor versions:
>> "v21 symbols are added and v20 symbols are modified, support for v20 ABI continues."
>> I am not sure if we can call the symbols added in minor versions as v21 ABI,
>> because it implies ABI compatibility is not required for them.
>>
>> 2) In ABI versioning [2], documented as .map files will have 'DPDK_20' and
>> 'DPDK_21' blocks.
>> But instead, I think they should have 'DPDK_20.0', 'DPDK_20.1', ... blocks, and
>> when major ABI version changed they all can be flattened to 'DPDK_21.0'.
>> For example we can't do ABI versioning between 20.2 & 20.3 if we don't have
>> these blocks.
>> Current block names in .map files are already defined as 'DPDK_20.0', what we
>> need to do is update the document to use 'DPDK_20.x' for the symbols added in
>> minor version and follow that process.
>>
> 
> What do we really gain from making such a change from the policy? I think
> it will work fine as-is, with putting all new symbols in the DPDK_21
> section. Whatever way you look at it, the functions will be forward but not
> backward compatible, which is all that really matters.
> 

As mentioned above it allows us ABI versioning between minor versions.

Also clarifies the intention, how current process puts can be read as ABI_21
APIs can be changed until 21 becomes supported ABI version. (because only ABI_20
is ABI compatible), so the question I asked in this thread no needed to be asked
and will be obvious.