[PATCH v3 1/1] mbuf: add optional no-copy dynamic field storage

[Konstantin Ananyev]

> Konstantin,
> Does this path work for you?
Yes, it does.
I just replied to your previous mail, probably our mails collided.
Konstantin

> -rt
>
> *From: *Morten Brørup <mb at smartsharesystems.com>
> *Date: *Tuesday, September 29, 2026 at 11:20 AM
> *To: *Randy Tice (rtice) <rtice at cisco.com>; Konstantin Ananyev 
> <konstantin.v.ananyev at yandex.ru>; dev at dpdk.org <dev at dpdk.org>
> *Cc: *Bruce Richardson <bruce.richardson at intel.com>; Harman Kalra 
> <hkalra at marvell.com>; Stephen Hemminger <stephen at networkplumber.org>
> *Subject: *RE: [PATCH v3 1/1] mbuf: add optional no-copy dynamic field 
> storage
>
> Randy,
>
> This is very close to what I suggested you explore.
>
> But one piece is missing:
>
> Registering fields in this metadata area should be managed through the 
> dynamic mbuf fields API.
>
> Without a central registry, only one module can use the new metadata 
> area; it cannot be used by multiple modules without coordination.
>
> And instead of rolling your own registry of fields in the metadata 
> area, just reuse the dynamic mbuf fields machinery.
>
> I agree with your proposed mbuf layout.
>
> There will be a performance cost for accessing the mbuf private data: 
> rte_mbuf_to_priv() will change from adding a simple constant offset 
> (sizeof(struct rte_mbuf)) to adding the value of a global variable 
> holding the offset, reflecting the startup-time configured metadata 
> area size.
>
> The global variable will be hot in the cache when working on mbuf 
> bursts, so I think this performance cost will be insignificant.
>
> Venlig hilsen / Kind regards,
>
> -Morten Brørup
>
> *From:* Randy Tice (rtice) [mailto:rtice at cisco.com]
> *Sent:* Tuesday, 29 September 2026 16.45
> *To:* Konstantin Ananyev; Morten Brørup; dev at dpdk.org
> *Cc:* Bruce Richardson; Harman Kalra; Stephen Hemminger
> *Subject:* Re: [PATCH v3 1/1] mbuf: add optional no-copy dynamic field 
> storage
>
>   Hi all,
>
>   Thanks for the discussion. We are now where I had hoped we’d get to 
> during
>
>   RFC but we are here.
>
> Konstantin, I understand your concern about making sizeof(struct rte_mbuf)
>
>   depend on a build-time option. That can create different mbuf 
> layouts between
>
>   DPDK builds that otherwise present the same ABI/version, which is 
> not a good
>
>   property for a core public structure.
>
>   After thinking through this again, I think the current patch may be 
> trying too
>
>   hard to make this a dynamic-field allocator feature. The actual 
> requirement is
>
>   simpler: a fixed global per-mbuf metadata area that is present in every
>
>   pktmbuf object, separate from ordinary application private data, and not
>
>   copied by mbuf copy/clone helpers.
>
>   The mbuf structure change would look roughly like this:
>
>    struct rte_mbuf {
>
> ...
>
> uint32_t dynfield1[9]; /**< Reserved for dynamic fields. */
>
>   +
>
>   + alignas(RTE_CACHE_LINE_SIZE)
>
>   + uint8_t metadata[];
>
>   + /**< Optional cache-line-aligned per-mbuf metadata area. */
>
>    };
>
>   Since this is a flexible array member, it does not change sizeof(struct
>
> rte_mbuf). The object layout would become:
>
>   struct rte_mbuf fixed header
>
>   global per-mbuf metadata area
>
> application private data
>
>   packet data buffer
>
>   With that layout, this could be sized at EAL init time rather than 
> by a build
>
>   option, for example:
>
> --mbuf-metadata-size=256
>
>   That avoids creating different DPDK builds with different mbuf 
> struct sizes or
>
> different build-time ABI expectations. The configured size would be 
> part of
>
>   the process/runtime configuration instead of requiring applications,
>
> libraries, and package providers to agree on a compile-time define.
>
>   The official mbuf helpers would account for this area before ordinary
>
> priv_size, so application private data remains available and does not 
> overlap
>
>   with the global metadata area.
>
>   This would also avoid changing the existing dynamic-field allocator 
> and copy
>
> semantics. The area would not be part of the dynamic-field registry; 
> it would
>
>   be explicit per-mbuf metadata storage for applications that deliberately
>
>   enable it.
>
>   That seems to address the main concerns:
>
>   - sizeof(struct rte_mbuf) remains fixed for ABI purposes.
>
>   - the metadata area is globally present across pktmbuf pools when 
> enabled.
>
>   - ordinary priv_size remains separate and available.
>
>   - dynamic-field allocator/copy behavior remains unchanged.
>
>   - users that do not enable the EAL option pay no extra per-mbuf 
> storage cost.
>
>   - applications do not need to be built against a different mbuf-size 
> define.
>
>   If this direction is acceptable, I can take a look at what it means in
>
>   practice for EAL configuration, mbuf layout helpers, pool 
> constructors, and
>
>   places that currently do direct object-layout math.
>
>   Thanks,
>
>   -rt
>
> *From: *Konstantin Ananyev <konstantin.v.ananyev at yandex.ru>
> *Date: *Tuesday, September 29, 2026 at 10:14 AM
> *To: *Morten Brørup <mb at smartsharesystems.com>; Randy Tice (rtice) 
> <rtice at cisco.com>; dev at dpdk.org <dev at dpdk.org>
> *Cc: *Bruce Richardson <bruce.richardson at intel.com>; Harman Kalra 
> <hkalra at marvell.com>; Stephen Hemminger <stephen at networkplumber.org>
> *Subject: *Re: [PATCH v3 1/1] mbuf: add optional no-copy dynamic field 
> storage
>
>
>
> >> From: Konstantin Ananyev [_mailto:konstantin.v.ananyev at yandex.ru_ 
> <mailto:konstantin.v.ananyev at yandex.ru>]
> >> Sent: Tuesday, 29 September 2026 15.13
> >>
> >> 29.09.2026 13:44, Morten Brørup пишет:
> >>>> From: Konstantin Ananyev [_mailto:konstantin.v.ananyev at yandex.ru_ 
> <mailto:konstantin.v.ananyev at yandex.ru>]
> >>>> Sent: Tuesday, 29 September 2026 14.00
> >>>>
> >>>> 28.09.2026 19:16, Randy L Tice пишет:
> >>>>> From: Randy L Tice <rtice at cisco.com>
> >>>>> Date: Thu, 03 Sep 2026 09:13:28 -0400
> >>>>>
> >>>>> Add build-time support for optional cache-line-aligned dynamic-
> >> field
> >>>>> storage at the end of struct rte_mbuf.
> >>>>>
> >>>>> The mbuf_dynfield3_size Meson option sets RTE_MBUF_DYNFIELD3_SIZE
> >> in
> >>>>> rte_build_config.h. A non-zero value enables the extra area and
> >> grows
> >>>>> every mbuf by the configured amount.
> >>>> I am strongly opposed to that patch.
> >>>> Inside mbuf we already do have priv_size that allows user to store
> >>>> his/her specific
> >>>> data straight after rte_mbuf in adjacent manner.
> >>>> It worked well so far for many use-cases (including VPP) and I don't
> >>>> see any
> >>>> reason why this is not enough.
> >>>>    From other side - making size of core rte_mbuf configurable at
> >> run-
> >>>> time,
> >>>> will affect DPDK ABI stability in a negative way.
> >>>> Fro my perspective it is much plausible in terms of ABI stability
> >> and
> >>>> predictability
> >>>> to have just one fixed layout for the mbuf.
> >>>> Konstantin
> >>> The private data area (priv_size) is independent per mbuf pool, and
> >> selected at run-time when creating each pool. As Randy explained in the
> >> RFC, this is unavailable for mbuf pools created by other components.
> >>
> >> I think it should be trivial to enforce minimal priv_size across all
> >> mbuf pools what will be obeyed by different components
> >> (as long as they do use rte_pktmbuf_pool_create() and friends):
> >> 1) introduce new EAL parameter 'mbuf-min-priv-size' or so (keep default
> >> as zero)
> >> 2) make rte_pktmbuf_pool_create_by_ops() and
> >> rte_pktmbuf_pool_create_extbuf() to check that input paramter
> >> 'priv_size' GE then value specified by EAL parameter, if so then return
> >> an error.
> > The private data area cannot be used.
> > Let's say one module creates an mbuf pool with priv_size of 8, and 
> uses those 8 bytes,
> > and some second module creates an mbuf pool with priv_size of 16, 
> and uses those 16 bytes.
> >
> > How should a module (or the application) know at which offset to 
> store its private data without overwriting the private data of other 
> modules?
> >
> > The mbuf dynamic field's registry manages centrally where each 
> module should store its own data, and the data is even accessible by 
> other modules (because they can fetch the offset to the data from the 
> registry)
> ok, I see, you need an ability to register/unregister/query layout for
> that private buffer (what we have now for dynfields).
> Then yes, if we'll add an ability to expand mbuf dynfield[] buffer that
> might be useful, and probably will become
> more popular then current 'priv_size' apporach.
> But I believe it shouldn't be a build time option.
> >
> >>> Mbuf dynamic fields are shared across all mbuf pools, and serves the
> >> need with an existing API. So I am strongly in favor of using the mbuf
> >> dynamic fields API for this.
> >>> I agree with Konstantin that it would be optimal if the size of the
> >> added dynfields area was run-time configurable (as an EAL startup
> >> parameter).
> >>> However, such a modification to the mbuf library would also require
> >> that the performance cost in the dataplane is negligible. We don't want
> >> to compromise on mbuf performance for applications not using this new
> >> feature.
> >>> Randy,
> >>> Could you please explore such an approach?
> >>>
> >>> -Morten
>