[dpdk-dev] [ovs-dev] [PATCH RFC] dpif-netdev: Add support Intel DPDK based ports.

[Thomas Graf]

Vincent,

On 01/29/2014 11:26 AM, Vincent JARDIN wrote:
> DPDK's ABIs are not Kernel's ABIs, they are not POSIX, there is no
> standard. Currently, there is no such plan to have a stable ABI since we
> need to keep freedom to chase CPU cycles over having a stable ABI. For
> instance, some applications on top of the DPDK process the packets in
> less than 150 CPU cycles (have a look at testpmd:
>    http://dpdk.org/browse/dpdk/tree/app/test-pmd )

I understand the requirement to not introduce overhead with wrappers
or shim layers. No problem with that. I believe this is mainly a policy
and release process issue.

Without a concept of stable interfaces, it will be difficult to
package and distribute RTE libraries, PMD, and DPDK applications. Right
now, the obvious path would include packaging the PMD bits together
with each DPDK application depending on the version of DPDK the binary
was compiled against. This is clearly not ideal.

> I agree that some areas could be improved since they are not into the
> critical datapath of packets, but still other areas remain very CPU
> constraints. For instance:
> http://dpdk.org/browse/dpdk/commit/lib/librte_ether/rte_ethdev.h?id=c3d0564cf0f00c3c9a61cf72bd4bd1c441740637
>
> is bad:
>     struct eth_dev_ops
> is churned, no comment, and a #ifdef that changes the structure
> according to compilation!

This is a very good example as it outlines the difference between
control structures and the fast path. We have this same exact trade off
in the kernel a lot where we have highly optimized internal APIs
towards modules and drivers but want to provide binary compatibility to
a certain extend.

As for the specific example you mention, it is relatively trivial to
make eth_dev_ops backwards compatible by appending appropriate padding
to the struct before a new major release and ensure that new members
are added by replacing the padding accordingly. Obviously no ifdefs
would be allowed anymore.

> Should an application use the librte libraries of the DPDK:
>    - you can use RTE_VERSION and RTE_VERSION_NUM :
> http://dpdk.org/doc/api/rte__version_8h.html#a8775053b0f721b9fa0457494cfbb7ed9

Right. This would be more or less identical to requiring a specific
DPDK version in OVS_CHEC_DPDK. It's not ideal to require application to
clutter their code with #ifdefs all over for every new minor release
though.

>    - you can write your own wrapper (with CPU overhead) in order to have
> a stable ABI, that wrapper should be tight to the versions of the librte
> => the overhead is part of your application instead of the DPDK,
>    - *otherwise recompile your software, it is opensource, what's the
> issue?*
>
> We are opened to any suggestion to have stable ABI, but it should never
> remove the options to have fast/efficient/compilation/CPU execution
> processing.

Absolutely agreed. We also don't want to add tons of abstraction and
overcomplicate everything. Still, I strongly believe that the definition
of stable interfaces towards applications and especially PMD is
essential.

I'm not proposing to standardize all the APIs towards applications on
the level of POSIX. DPDK is in early stages and disruptive changes will
come along. What I would propose on an abstract level is:

1. Extend but not break API between minor releases. Postpone API
    breakages to the next major release. High cadence of major
    releases initially, lower cadence as DPDK matures.

2. Define ABI stability towards PMD for minor releases to allow
    isolated packaging of PMD by padding control structures and keeping
    functions ABI stable.

I realize that this might be less trivial than it seems without
sacrificing performance but I consider it effort well spent.

Thomas