[PATCH v3 1/1] mbuf: add optional no-copy dynamic field storage

[Randy Tice (rtice)]

Konstantin,
Does this path work for you?
-rt

From: Morten Brørup <mb at smartsharesystems.com>
Date: Tuesday, September 29, 2026 at 11:20 AM
To: Randy Tice (rtice) <rtice at cisco.com>; Konstantin Ananyev <konstantin.v.ananyev at yandex.ru>; dev at dpdk.org <dev at dpdk.org>
Cc: Bruce Richardson <bruce.richardson at intel.com>; Harman Kalra <hkalra at marvell.com>; Stephen Hemminger <stephen at networkplumber.org>
Subject: RE: [PATCH v3 1/1] mbuf: add optional no-copy dynamic field storage

Randy,

This is very close to what I suggested you explore.

But one piece is missing:
Registering fields in this metadata area should be managed through the dynamic mbuf fields API.
Without a central registry, only one module can use the new metadata area; it cannot be used by multiple modules without coordination.
And instead of rolling your own registry of fields in the metadata area, just reuse the dynamic mbuf fields machinery.


I agree with your proposed mbuf layout.

There will be a performance cost for accessing the mbuf private data: rte_mbuf_to_priv() will change from adding a simple constant offset (sizeof(struct rte_mbuf)) to adding the value of a global variable holding the offset, reflecting the startup-time configured metadata area size.
The global variable will be hot in the cache when working on mbuf bursts, so I think this performance cost will be insignificant.


Venlig hilsen / Kind regards,
-Morten Brørup

From: Randy Tice (rtice) [mailto:rtice at cisco.com]
Sent: Tuesday, 29 September 2026 16.45
To: Konstantin Ananyev; Morten Brørup; dev at dpdk.org
Cc: Bruce Richardson; Harman Kalra; Stephen Hemminger
Subject: Re: [PATCH v3 1/1] mbuf: add optional no-copy dynamic field storage

  Hi all,

  Thanks for the discussion. We are now where I had hoped we’d get to during
  RFC but we are here.

  Konstantin, I understand your concern about making sizeof(struct rte_mbuf)
  depend on a build-time option. That can create different mbuf layouts between
  DPDK builds that otherwise present the same ABI/version, which is not a good
  property for a core public structure.

  After thinking through this again, I think the current patch may be trying too
  hard to make this a dynamic-field allocator feature. The actual requirement is
  simpler: a fixed global per-mbuf metadata area that is present in every
  pktmbuf object, separate from ordinary application private data, and not
  copied by mbuf copy/clone helpers.

  The mbuf structure change would look roughly like this:

   struct rte_mbuf {
          ...
          uint32_t dynfield1[9]; /**< Reserved for dynamic fields. */
  +
  +       alignas(RTE_CACHE_LINE_SIZE)
  +       uint8_t metadata[];
  +       /**< Optional cache-line-aligned per-mbuf metadata area. */
   };

  Since this is a flexible array member, it does not change sizeof(struct
  rte_mbuf). The object layout would become:

  struct rte_mbuf fixed header
  global per-mbuf metadata area
  application private data
  packet data buffer

  With that layout, this could be sized at EAL init time rather than by a build
  option, for example:

  --mbuf-metadata-size=256

  That avoids creating different DPDK builds with different mbuf struct sizes or
  different build-time ABI expectations. The configured size would be part of
  the process/runtime configuration instead of requiring applications,
  libraries, and package providers to agree on a compile-time define.

  The official mbuf helpers would account for this area before ordinary
  priv_size, so application private data remains available and does not overlap
  with the global metadata area.

  This would also avoid changing the existing dynamic-field allocator and copy
  semantics. The area would not be part of the dynamic-field registry; it would
  be explicit per-mbuf metadata storage for applications that deliberately
  enable it.

  That seems to address the main concerns:

  - sizeof(struct rte_mbuf) remains fixed for ABI purposes.
  - the metadata area is globally present across pktmbuf pools when enabled.
  - ordinary priv_size remains separate and available.
  - dynamic-field allocator/copy behavior remains unchanged.
  - users that do not enable the EAL option pay no extra per-mbuf storage cost.
  - applications do not need to be built against a different mbuf-size define.

  If this direction is acceptable, I can take a look at what it means in
  practice for EAL configuration, mbuf layout helpers, pool constructors, and
  places that currently do direct object-layout math.

  Thanks,
  -rt

From: Konstantin Ananyev <konstantin.v.ananyev at yandex.ru>
Date: Tuesday, September 29, 2026 at 10:14 AM
To: Morten Brørup <mb at smartsharesystems.com>; Randy Tice (rtice) <rtice at cisco.com>; dev at dpdk.org <dev at dpdk.org>
Cc: Bruce Richardson <bruce.richardson at intel.com>; Harman Kalra <hkalra at marvell.com>; Stephen Hemminger <stephen at networkplumber.org>
Subject: Re: [PATCH v3 1/1] mbuf: add optional no-copy dynamic field storage


>> From: Konstantin Ananyev [mailto:konstantin.v.ananyev at yandex.ru]
>> Sent: Tuesday, 29 September 2026 15.13
>>
>> 29.09.2026 13:44, Morten Brørup пишет:
>>>> From: Konstantin Ananyev [mailto:konstantin.v.ananyev at yandex.ru]
>>>> Sent: Tuesday, 29 September 2026 14.00
>>>>
>>>> 28.09.2026 19:16, Randy L Tice пишет:
>>>>> From: Randy L Tice <rtice at cisco.com>
>>>>> Date: Thu, 03 Sep 2026 09:13:28 -0400
>>>>>
>>>>> Add build-time support for optional cache-line-aligned dynamic-
>> field
>>>>> storage at the end of struct rte_mbuf.
>>>>>
>>>>> The mbuf_dynfield3_size Meson option sets RTE_MBUF_DYNFIELD3_SIZE
>> in
>>>>> rte_build_config.h. A non-zero value enables the extra area and
>> grows
>>>>> every mbuf by the configured amount.
>>>> I am strongly opposed to that patch.
>>>> Inside mbuf we already do have priv_size that allows user to store
>>>> his/her specific
>>>> data straight after rte_mbuf in adjacent manner.
>>>> It worked well so far for many use-cases (including VPP) and I don't
>>>> see any
>>>> reason why this is not enough.
>>>>    From other side - making size of core rte_mbuf configurable at
>> run-
>>>> time,
>>>> will affect DPDK ABI stability in a negative way.
>>>> Fro my perspective it is much plausible in terms of ABI stability
>> and
>>>> predictability
>>>> to have just one fixed layout for the mbuf.
>>>> Konstantin
>>> The private data area (priv_size) is independent per mbuf pool, and
>> selected at run-time when creating each pool. As Randy explained in the
>> RFC, this is unavailable for mbuf pools created by other components.
>>
>> I think it should be trivial to enforce minimal priv_size across all
>> mbuf pools what will be obeyed by different components
>> (as long as they do use rte_pktmbuf_pool_create() and friends):
>> 1) introduce new EAL parameter 'mbuf-min-priv-size' or so (keep default
>> as zero)
>> 2) make rte_pktmbuf_pool_create_by_ops() and
>> rte_pktmbuf_pool_create_extbuf() to check that input paramter
>> 'priv_size' GE then value specified by EAL parameter, if so then return
>> an error.
> The private data area cannot be used.
> Let's say one module creates an mbuf pool with priv_size of 8, and uses those 8 bytes,
> and some second module creates an mbuf pool with priv_size of 16, and uses those 16 bytes.
>
> How should a module (or the application) know at which offset to store its private data without overwriting the private data of other modules?
>
> The mbuf dynamic field's registry manages centrally where each module should store its own data, and the data is even accessible by other modules (because they can fetch the offset to the data from the registry)
ok, I see, you need an ability to register/unregister/query layout for
that private buffer (what we have now for dynfields).
Then yes, if we'll add an ability to expand mbuf dynfield[] buffer that
might be useful, and probably will become
more popular then current 'priv_size' apporach.
But I believe it shouldn't be a build time option.
>
>>> Mbuf dynamic fields are shared across all mbuf pools, and serves the
>> need with an existing API. So I am strongly in favor of using the mbuf
>> dynamic fields API for this.
>>> I agree with Konstantin that it would be optimal if the size of the
>> added dynfields area was run-time configurable (as an EAL startup
>> parameter).
>>> However, such a modification to the mbuf library would also require
>> that the performance cost in the dataplane is negligible. We don't want
>> to compromise on mbuf performance for applications not using this new
>> feature.
>>> Randy,
>>> Could you please explore such an approach?
>>>
>>> -Morten
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mails.dpdk.org/archives/dev/attachments/20260929/e57e0506/attachment-0001.htm>