[dpdk-dev] [PATCH v4 0/7] hide eth dev related structures

[Ali Alnubani]

> -----Original Message-----
> From: Ali Alnubani
> Sent: Wednesday, October 6, 2021 7:43 PM
> To: Konstantin Ananyev <konstantin.ananyev at intel.com>; dev at dpdk.org
> Cc: xiaoyun.li at intel.com; anoobj at marvell.com; jerinj at marvell.com;
> ndabilpuram at marvell.com; adwivedi at marvell.com;
> shepard.siegel at atomicrules.com; ed.czeck at atomicrules.com;
> john.miller at atomicrules.com; irusskikh at marvell.com;
> ajit.khaparde at broadcom.com; somnath.kotur at broadcom.com;
> rahul.lakkireddy at chelsio.com; hemant.agrawal at nxp.com;
> sachin.saxena at oss.nxp.com; haiyue.wang at intel.com; johndale at cisco.com;
> hyonkim at cisco.com; qi.z.zhang at intel.com; xiao.w.wang at intel.com;
> humin29 at huawei.com; yisen.zhuang at huawei.com; oulijun at huawei.com;
> beilei.xing at intel.com; jingjing.wu at intel.com; qiming.yang at intel.com;
> Matan Azrad <matan at nvidia.com>; Slava Ovsiienko
> <viacheslavo at nvidia.com>; sthemmin at microsoft.com; NBU-Contact-longli
> <longli at microsoft.com>; heinrich.kuhn at corigine.com;
> kirankumark at marvell.com; andrew.rybchenko at oktetlabs.ru;
> mczekaj at marvell.com; jiawenwu at trustnetic.com;
> jianwang at trustnetic.com; maxime.coquelin at redhat.com;
> chenbo.xia at intel.com; NBU-Contact-Thomas Monjalon
> <thomas at monjalon.net>; ferruh.yigit at intel.com; mdr at ashroe.eu;
> jay.jayatheerthan at intel.com
> Subject: RE: [dpdk-dev] [PATCH v4 0/7] hide eth dev related structures
> 
> > -----Original Message-----
> > From: dev <dev-bounces at dpdk.org> On Behalf Of Konstantin Ananyev
> > Sent: Monday, October 4, 2021 4:56 PM
> > To: dev at dpdk.org
> > Cc: xiaoyun.li at intel.com; anoobj at marvell.com; jerinj at marvell.com;
> > ndabilpuram at marvell.com; adwivedi at marvell.com;
> > shepard.siegel at atomicrules.com; ed.czeck at atomicrules.com;
> > john.miller at atomicrules.com; irusskikh at marvell.com;
> > ajit.khaparde at broadcom.com; somnath.kotur at broadcom.com;
> > rahul.lakkireddy at chelsio.com; hemant.agrawal at nxp.com;
> > sachin.saxena at oss.nxp.com; haiyue.wang at intel.com;
> johndale at cisco.com;
> > hyonkim at cisco.com; qi.z.zhang at intel.com; xiao.w.wang at intel.com;
> > humin29 at huawei.com; yisen.zhuang at huawei.com; oulijun at huawei.com;
> > beilei.xing at intel.com; jingjing.wu at intel.com; qiming.yang at intel.com;
> > Matan Azrad <matan at nvidia.com>; Slava Ovsiienko
> > <viacheslavo at nvidia.com>; sthemmin at microsoft.com; NBU-Contact-longli
> > <longli at microsoft.com>; heinrich.kuhn at corigine.com;
> > kirankumark at marvell.com; andrew.rybchenko at oktetlabs.ru;
> > mczekaj at marvell.com; jiawenwu at trustnetic.com;
> jianwang at trustnetic.com;
> > maxime.coquelin at redhat.com; chenbo.xia at intel.com; NBU-Contact-
> Thomas
> > Monjalon <thomas at monjalon.net>; ferruh.yigit at intel.com;
> mdr at ashroe.eu;
> > jay.jayatheerthan at intel.com; Konstantin Ananyev
> > <konstantin.ananyev at intel.com>
> > Subject: [dpdk-dev] [PATCH v4 0/7] hide eth dev related structures
> >
> > v4 changes:
> >  - Fix secondary process attach (Pavan)
> >  - Fix build failure (Ferruh)
> >  - Update lib/ethdev/verion.map (Ferruh)
> >    Note that moving newly added symbols from EXPERIMENTAL to DPDK_22
> >    section makes checkpatch.sh to complain.
> >
> > v3 changes:
> >  - Changes in public struct naming (Jerin/Haiyue)
> >  - Split patches
> >  - Update docs
> >  - Shamelessly included Andrew's patch:
> >
> > https://patches.dpdk.org/project/dpdk/patch/20210928154856.1015020-1-
> > andrew.rybchenko at oktetlabs.ru/
> >    into these series.
> >    I have to do similar thing here, so decided to avoid duplicated effort.
> >
> > The aim of these patch series is to make rte_ethdev core data
> > structures (rte_eth_dev, rte_eth_dev_data, rte_eth_rxtx_callback,
> > etc.) internal to DPDK and not visible to the user.
> > That should allow future possible changes to core ethdev related
> > structures to be transparent to the user and help to improve ABI/API
> stability.
> > Note that current ethdev API is preserved, but it is a formal ABI break.
> >
> > The work is based on previous discussions at:
> > https://www.mail-archive.com/dev@dpdk.org/msg211405.html
> > https://www.mail-archive.com/dev@dpdk.org/msg216685.html
> > and consists of the following main points:
> > 1. Copy public 'fast' function pointers (rx_pkt_burst(), etc.) and
> >    related data pointer from rte_eth_dev into a separate flat array.
> >    We keep it public to still be able to use inline functions for these
> >    'fast' calls (like rte_eth_rx_burst(), etc.) to avoid/minimize slowdown.
> >    Note that apart from function pointers itself, each element of this
> >    flat array also contains two opaque pointers for each ethdev:
> >    1) a pointer to an array of internal queue data pointers
> >    2)  points to array of queue callback data pointers.
> >    Note that exposing this extra information allows us to avoid extra
> >    changes inside PMD level, plus should help to avoid possible
> >    performance degradation.
> > 2. Change implementation of 'fast' inline ethdev functions
> >    (rte_eth_rx_burst(), etc.) to use new public flat array.
> >    While it is an ABI breakage, this change is intended to be transparent
> >    for both users (no changes in user app is required) and PMD developers
> >    (no changes in PMD is required).
> >    One extra note - with new implementation RX/TX callback invocation
> >    will cost one extra function call with this changes. That might cause
> >    some slowdown for code-path with RX/TX callbacks heavily involved.
> >    Hope such trade-off is acceptable for the community.
> > 3. Move rte_eth_dev, rte_eth_dev_data, rte_eth_rxtx_callback and
> related
> >    things into internal header: <ethdev_driver.h>.
> >
> > That approach was selected to:
> >   - Avoid(/minimize) possible performance losses.
> >   - Minimize required changes inside PMDs.
> >
> > Performance testing results (ICX 2.0GHz, E810 (ice)):
> >  - testpmd macswap fwd mode, plus
> >    a) no RX/TX callbacks:
> >       no actual slowdown observed
> >    b) bpf-load rx 0 0 JM ./dpdk.org/examples/bpf/t3.o:
> >       ~2% slowdown
> >  - l3fwd: no actual slowdown observed
> >
> > Would like to thank everyone who already reviewed and tested previous
> > versions of these series. All other interested parties please don't be
> > shy and provide your feedback.
> >
> > Konstantin Ananyev (7):
> >   ethdev: allocate max space for internal queue array
> >   ethdev: change input parameters for rx_queue_count
> >   ethdev: copy ethdev 'fast' API into separate structure
> >   ethdev: make burst functions to use new flat array
> >   ethdev: add API to retrieve multiple ethernet addresses
> >   ethdev: remove legacy Rx descriptor done API
> >   ethdev: hide eth dev related structures
> >
> 
> Tested single and multi-core packet forwarding performance with testpmd
> on both ConnectX-5 and ConnectX-6 Dx.
> 

I should have mentioned that I didn't see any noticeable regressions with the cases I mentioned.

> Tested-by: Ali Alnubani <alialnu at nvidia.com>
> 
> Thanks,
> Ali