<html xmlns:v="urn:schemas-microsoft-com:vml" xmlns:o="urn:schemas-microsoft-com:office:office" xmlns:w="urn:schemas-microsoft-com:office:word" xmlns:m="http://schemas.microsoft.com/office/2004/12/omml" xmlns="http://www.w3.org/TR/REC-html40"><head><meta http-equiv=Content-Type content="text/html; charset=utf-8"><meta name=Generator content="Microsoft Word 12 (filtered medium)"><style></style></head><body lang=EN-US link=blue vlink=purple><div class=WordSection1><p class=MsoNormal><a name="_MailEndCompose"><span style='font-size:11.0pt;font-family:"Calibri","sans-serif";color:#1F497D'>Randy,<o:p></o:p></span></a></p><p class=MsoNormal><span style='font-size:11.0pt;font-family:"Calibri","sans-serif";color:#1F497D'><o:p> </o:p></span></p><p class=MsoNormal><span style='font-size:11.0pt;font-family:"Calibri","sans-serif";color:#1F497D'>This is very close to what I suggested you explore.<o:p></o:p></span></p><p class=MsoNormal><span style='font-size:11.0pt;font-family:"Calibri","sans-serif";color:#1F497D'><o:p> </o:p></span></p><p class=MsoNormal><span style='font-size:11.0pt;font-family:"Calibri","sans-serif";color:#1F497D'>But one piece is missing:<o:p></o:p></span></p><p class=MsoNormal><span style='font-size:11.0pt;font-family:"Calibri","sans-serif";color:#1F497D'>Registering fields in this metadata area should be managed through the dynamic mbuf fields API.<o:p></o:p></span></p><p class=MsoNormal><span style='font-size:11.0pt;font-family:"Calibri","sans-serif";color:#1F497D'>Without a central registry, only one module can use the new metadata area; it cannot be used by multiple modules without coordination.<o:p></o:p></span></p><p class=MsoNormal><span style='font-size:11.0pt;font-family:"Calibri","sans-serif";color:#1F497D'>And instead of rolling your own registry of fields in the metadata area, just reuse the dynamic mbuf fields machinery.<o:p></o:p></span></p><p class=MsoNormal><span style='font-size:11.0pt;font-family:"Calibri","sans-serif";color:#1F497D'><o:p> </o:p></span></p><p class=MsoNormal><span style='font-size:11.0pt;font-family:"Calibri","sans-serif";color:#1F497D'><o:p> </o:p></span></p><p class=MsoNormal><span style='font-size:11.0pt;font-family:"Calibri","sans-serif";color:#1F497D'>I agree with your proposed mbuf layout.<o:p></o:p></span></p><p class=MsoNormal><span style='font-size:11.0pt;font-family:"Calibri","sans-serif";color:#1F497D'><o:p> </o:p></span></p><p class=MsoNormal><span style='font-size:11.0pt;font-family:"Calibri","sans-serif";color:#1F497D'>There will be a performance cost for accessing the mbuf private data: rte_mbuf_to_priv() will change from adding a simple constant offset (sizeof(struct rte_mbuf)) to adding the value of a global variable holding the offset, reflecting the startup-time configured metadata area size.<o:p></o:p></span></p><p class=MsoNormal><span style='font-size:11.0pt;font-family:"Calibri","sans-serif";color:#1F497D'>The global variable will be hot in the cache when working on mbuf bursts, so I think this performance cost will be insignificant.<o:p></o:p></span></p><p class=MsoNormal><span style='font-size:11.0pt;font-family:"Calibri","sans-serif";color:#1F497D'><o:p> </o:p></span></p><p class=MsoNormal><span style='font-size:11.0pt;font-family:"Calibri","sans-serif";color:#1F497D'><o:p> </o:p></span></p><div><p class=MsoNormal><span style='font-size:11.0pt;font-family:"Calibri","sans-serif";color:#1F497D'>Venlig hilsen / Kind regards,<o:p></o:p></span></p><p class=MsoNormal><span style='font-size:11.0pt;font-family:"Calibri","sans-serif";color:#1F497D'>-Morten Brørup<o:p></o:p></span></p></div><p class=MsoNormal><span style='font-size:11.0pt;font-family:"Calibri","sans-serif";color:#1F497D'><o:p> </o:p></span></p><div style='border:none;border-left:solid blue 1.5pt;padding:0cm 0cm 0cm 4.0pt'><div><div style='border:none;border-top:solid #B5C4DF 1.0pt;padding:3.0pt 0cm 0cm 0cm'><p class=MsoNormal><b><span style='font-size:10.0pt;font-family:"Tahoma","sans-serif"'>From:</span></b><span style='font-size:10.0pt;font-family:"Tahoma","sans-serif"'> Randy Tice (rtice) [mailto:rtice@cisco.com] <br><b>Sent:</b> Tuesday, 29 September 2026 16.45<br><b>To:</b> Konstantin Ananyev; Morten Brørup; dev@dpdk.org<br><b>Cc:</b> Bruce Richardson; Harman Kalra; Stephen Hemminger<br><b>Subject:</b> Re: [PATCH v3 1/1] mbuf: add optional no-copy dynamic field storage<o:p></o:p></span></p></div></div><p class=MsoNormal><o:p> </o:p></p><div><p class=MsoNormal><span style='font-family:Aptos;color:black'> Hi all,<o:p></o:p></span></p></div><div><p class=MsoNormal><span style='font-family:Aptos;color:black'><o:p> </o:p></span></p></div><div><p class=MsoNormal><span style='font-family:Aptos;color:black'> Thanks for the discussion. We are now where I had hoped we’d get to during<o:p></o:p></span></p></div><div><p class=MsoNormal><span style='font-family:Aptos;color:black'> RFC but we are here.<o:p></o:p></span></p></div><div><p class=MsoNormal><span style='font-family:Aptos;color:black'><o:p> </o:p></span></p></div><div><p class=MsoNormal><span style='font-family:Aptos;color:black'> Konstantin, I understand your concern about making sizeof(struct rte_mbuf)<o:p></o:p></span></p></div><div><p class=MsoNormal><span style='font-family:Aptos;color:black'> depend on a build-time option. That can create different mbuf layouts between<o:p></o:p></span></p></div><div><p class=MsoNormal><span style='font-family:Aptos;color:black'> DPDK builds that otherwise present the same ABI/version, which is not a good<o:p></o:p></span></p></div><div><p class=MsoNormal><span style='font-family:Aptos;color:black'> property for a core public structure.<o:p></o:p></span></p></div><div><p class=MsoNormal><span style='font-family:Aptos;color:black'><o:p> </o:p></span></p></div><div><p class=MsoNormal><span style='font-family:Aptos;color:black'> After thinking through this again, I think the current patch may be trying too<o:p></o:p></span></p></div><div><p class=MsoNormal><span style='font-family:Aptos;color:black'> hard to make this a dynamic-field allocator feature. The actual requirement is<o:p></o:p></span></p></div><div><p class=MsoNormal><span style='font-family:Aptos;color:black'> simpler: a fixed global per-mbuf metadata area that is present in every<o:p></o:p></span></p></div><div><p class=MsoNormal><span style='font-family:Aptos;color:black'> pktmbuf object, separate from ordinary application private data, and not<o:p></o:p></span></p></div><div><p class=MsoNormal><span style='font-family:Aptos;color:black'> copied by mbuf copy/clone helpers.<o:p></o:p></span></p></div><div><p class=MsoNormal><span style='font-family:Aptos;color:black'><o:p> </o:p></span></p></div><div><p class=MsoNormal><span style='font-family:Aptos;color:black'> The mbuf structure change would look roughly like this:<o:p></o:p></span></p></div><div><p class=MsoNormal><span style='font-family:Aptos;color:black'><o:p> </o:p></span></p></div><div><p class=MsoNormal><span style='font-family:Aptos;color:black'> struct rte_mbuf {<o:p></o:p></span></p></div><div><p class=MsoNormal><span style='font-family:Aptos;color:black'> ...<o:p></o:p></span></p></div><div><p class=MsoNormal><span style='font-family:Aptos;color:black'> uint32_t dynfield1[9]; /**< Reserved for dynamic fields. */<o:p></o:p></span></p></div><div><p class=MsoNormal><span style='font-family:Aptos;color:black'> +<o:p></o:p></span></p></div><div><p class=MsoNormal><span style='font-family:Aptos;color:black'> + alignas(RTE_CACHE_LINE_SIZE)<o:p></o:p></span></p></div><div><p class=MsoNormal><span style='font-family:Aptos;color:black'> + uint8_t metadata[];<o:p></o:p></span></p></div><div><p class=MsoNormal><span style='font-family:Aptos;color:black'> + /**< Optional cache-line-aligned per-mbuf metadata area. */<o:p></o:p></span></p></div><div><p class=MsoNormal><span style='font-family:Aptos;color:black'> };<o:p></o:p></span></p></div><div><p class=MsoNormal><span style='font-family:Aptos;color:black'><o:p> </o:p></span></p></div><div><p class=MsoNormal><span style='font-family:Aptos;color:black'> Since this is a flexible array member, it does not change sizeof(struct<o:p></o:p></span></p></div><div><p class=MsoNormal><span style='font-family:Aptos;color:black'> rte_mbuf). The object layout would become:<o:p></o:p></span></p></div><div><p class=MsoNormal><span style='font-family:Aptos;color:black'><o:p> </o:p></span></p></div><div><p class=MsoNormal><span style='font-family:Aptos;color:black'> struct rte_mbuf fixed header<o:p></o:p></span></p></div><div><p class=MsoNormal><span style='font-family:Aptos;color:black'> global per-mbuf metadata area<o:p></o:p></span></p></div><div><p class=MsoNormal><span style='font-family:Aptos;color:black'> application private data<o:p></o:p></span></p></div><div><p class=MsoNormal><span style='font-family:Aptos;color:black'> packet data buffer<o:p></o:p></span></p></div><div><p class=MsoNormal><span style='font-family:Aptos;color:black'><o:p> </o:p></span></p></div><div><p class=MsoNormal><span style='font-family:Aptos;color:black'> With that layout, this could be sized at EAL init time rather than by a build<o:p></o:p></span></p></div><div><p class=MsoNormal><span style='font-family:Aptos;color:black'> option, for example:<o:p></o:p></span></p></div><div><p class=MsoNormal><span style='font-family:Aptos;color:black'><o:p> </o:p></span></p></div><div><p class=MsoNormal><span style='font-family:Aptos;color:black'> --mbuf-metadata-size=256<o:p></o:p></span></p></div><div><p class=MsoNormal><span style='font-family:Aptos;color:black'><o:p> </o:p></span></p></div><div><p class=MsoNormal><span style='font-family:Aptos;color:black'> That avoids creating different DPDK builds with different mbuf struct sizes or<o:p></o:p></span></p></div><div><p class=MsoNormal><span style='font-family:Aptos;color:black'> different build-time ABI expectations. The configured size would be part of<o:p></o:p></span></p></div><div><p class=MsoNormal><span style='font-family:Aptos;color:black'> the process/runtime configuration instead of requiring applications,<o:p></o:p></span></p></div><div><p class=MsoNormal><span style='font-family:Aptos;color:black'> libraries, and package providers to agree on a compile-time define.<o:p></o:p></span></p></div><div><p class=MsoNormal><span style='font-family:Aptos;color:black'><o:p> </o:p></span></p></div><div><p class=MsoNormal><span style='font-family:Aptos;color:black'> The official mbuf helpers would account for this area before ordinary<o:p></o:p></span></p></div><div><p class=MsoNormal><span style='font-family:Aptos;color:black'> priv_size, so application private data remains available and does not overlap<o:p></o:p></span></p></div><div><p class=MsoNormal><span style='font-family:Aptos;color:black'> with the global metadata area.<o:p></o:p></span></p></div><div><p class=MsoNormal><span style='font-family:Aptos;color:black'><o:p> </o:p></span></p></div><div><p class=MsoNormal><span style='font-family:Aptos;color:black'> This would also avoid changing the existing dynamic-field allocator and copy<o:p></o:p></span></p></div><div><p class=MsoNormal><span style='font-family:Aptos;color:black'> semantics. The area would not be part of the dynamic-field registry; it would<o:p></o:p></span></p></div><div><p class=MsoNormal><span style='font-family:Aptos;color:black'> be explicit per-mbuf metadata storage for applications that deliberately<o:p></o:p></span></p></div><div><p class=MsoNormal><span style='font-family:Aptos;color:black'> enable it.<o:p></o:p></span></p></div><div><p class=MsoNormal><span style='font-family:Aptos;color:black'><o:p> </o:p></span></p></div><div><p class=MsoNormal><span style='font-family:Aptos;color:black'> That seems to address the main concerns:<o:p></o:p></span></p></div><div><p class=MsoNormal><span style='font-family:Aptos;color:black'><o:p> </o:p></span></p></div><div><p class=MsoNormal><span style='font-family:Aptos;color:black'> - sizeof(struct rte_mbuf) remains fixed for ABI purposes.<o:p></o:p></span></p></div><div><p class=MsoNormal><span style='font-family:Aptos;color:black'> - the metadata area is globally present across pktmbuf pools when enabled.<o:p></o:p></span></p></div><div><p class=MsoNormal><span style='font-family:Aptos;color:black'> - ordinary priv_size remains separate and available.<o:p></o:p></span></p></div><div><p class=MsoNormal><span style='font-family:Aptos;color:black'> - dynamic-field allocator/copy behavior remains unchanged.<o:p></o:p></span></p></div><div><p class=MsoNormal><span style='font-family:Aptos;color:black'> - users that do not enable the EAL option pay no extra per-mbuf storage cost.<o:p></o:p></span></p></div><div><p class=MsoNormal><span style='font-family:Aptos;color:black'> - applications do not need to be built against a different mbuf-size define.<o:p></o:p></span></p></div><div><p class=MsoNormal><span style='font-family:Aptos;color:black'><o:p> </o:p></span></p></div><div><p class=MsoNormal><span style='font-family:Aptos;color:black'> If this direction is acceptable, I can take a look at what it means in<o:p></o:p></span></p></div><div><p class=MsoNormal><span style='font-family:Aptos;color:black'> practice for EAL configuration, mbuf layout helpers, pool constructors, and<o:p></o:p></span></p></div><div><p class=MsoNormal><span style='font-family:Aptos;color:black'> places that currently do direct object-layout math.<o:p></o:p></span></p></div><div><p class=MsoNormal><span style='font-family:Aptos;color:black'><o:p> </o:p></span></p></div><div><p class=MsoNormal><span style='font-family:Aptos;color:black'> Thanks,<o:p></o:p></span></p></div><div><p class=MsoNormal><span style='font-family:Aptos;color:black'> -rt<o:p></o:p></span></p></div><div><p class=MsoNormal><span style='font-family:Aptos;color:black'><o:p> </o:p></span></p></div><div id=mail-editor-reference-message-container><div style='border:none;border-top:solid windowtext 1.0pt;padding:3.0pt 0cm 0cm 0cm;border-color:currentcolor currentcolor'><p class=MsoNormal style='margin-bottom:12.0pt'><b><span style='font-family:Aptos;color:black'>From: </span></b><span style='font-family:Aptos;color:black'>Konstantin Ananyev <konstantin.v.ananyev@yandex.ru><br><b>Date: </b>Tuesday, September 29, 2026 at 10:14 AM<br><b>To: </b>Morten Brørup <mb@smartsharesystems.com>; Randy Tice (rtice) <rtice@cisco.com>; dev@dpdk.org <dev@dpdk.org><br><b>Cc: </b>Bruce Richardson <bruce.richardson@intel.com>; Harman Kalra <hkalra@marvell.com>; Stephen Hemminger <stephen@networkplumber.org><br><b>Subject: </b>Re: [PATCH v3 1/1] mbuf: add optional no-copy dynamic field storage<o:p></o:p></span></p></div><div id=mail-editor-reference-message-body><div><p class=MsoNormal style='margin-bottom:12.0pt'><span style='font-size:11.0pt'><br><br>>> From: Konstantin Ananyev [<a href="mailto:konstantin.v.ananyev@yandex.ru">mailto:konstantin.v.ananyev@yandex.ru</a>]<br>>> Sent: Tuesday, 29 September 2026 15.13<br>>><br>>> 29.09.2026 13:44, Morten Brørup пишет:<br>>>>> From: Konstantin Ananyev [<a href="mailto:konstantin.v.ananyev@yandex.ru">mailto:konstantin.v.ananyev@yandex.ru</a>]<br>>>>> Sent: Tuesday, 29 September 2026 14.00<br>>>>><br>>>>> 28.09.2026 19:16, Randy L Tice пишет:<br>>>>>> From: Randy L Tice <rtice@cisco.com><br>>>>>> Date: Thu, 03 Sep 2026 09:13:28 -0400<br>>>>>><br>>>>>> Add build-time support for optional cache-line-aligned dynamic-<br>>> field<br>>>>>> storage at the end of struct rte_mbuf.<br>>>>>><br>>>>>> The mbuf_dynfield3_size Meson option sets RTE_MBUF_DYNFIELD3_SIZE<br>>> in<br>>>>>> rte_build_config.h. A non-zero value enables the extra area and<br>>> grows<br>>>>>> every mbuf by the configured amount.<br>>>>> I am strongly opposed to that patch.<br>>>>> Inside mbuf we already do have priv_size that allows user to store<br>>>>> his/her specific<br>>>>> data straight after rte_mbuf in adjacent manner.<br>>>>> It worked well so far for many use-cases (including VPP) and I don't<br>>>>> see any<br>>>>> reason why this is not enough.<br>>>>> From other side - making size of core rte_mbuf configurable at<br>>> run-<br>>>>> time,<br>>>>> will affect DPDK ABI stability in a negative way.<br>>>>> Fro my perspective it is much plausible in terms of ABI stability<br>>> and<br>>>>> predictability<br>>>>> to have just one fixed layout for the mbuf.<br>>>>> Konstantin<br>>>> The private data area (priv_size) is independent per mbuf pool, and<br>>> selected at run-time when creating each pool. As Randy explained in the<br>>> RFC, this is unavailable for mbuf pools created by other components.<br>>><br>>> I think it should be trivial to enforce minimal priv_size across all<br>>> mbuf pools what will be obeyed by different components<br>>> (as long as they do use rte_pktmbuf_pool_create() and friends):<br>>> 1) introduce new EAL parameter 'mbuf-min-priv-size' or so (keep default<br>>> as zero)<br>>> 2) make rte_pktmbuf_pool_create_by_ops() and<br>>> rte_pktmbuf_pool_create_extbuf() to check that input paramter<br>>> 'priv_size' GE then value specified by EAL parameter, if so then return<br>>> an error.<br>> The private data area cannot be used.<br>> Let's say one module creates an mbuf pool with priv_size of 8, and uses those 8 bytes,<br>> and some second module creates an mbuf pool with priv_size of 16, and uses those 16 bytes.<br>><br>> How should a module (or the application) know at which offset to store its private data without overwriting the private data of other modules?<br>><br>> The mbuf dynamic field's registry manages centrally where each module should store its own data, and the data is even accessible by other modules (because they can fetch the offset to the data from the registry)<br>ok, I see, you need an ability to register/unregister/query layout for<br>that private buffer (what we have now for dynfields).<br>Then yes, if we'll add an ability to expand mbuf dynfield[] buffer that<br>might be useful, and probably will become<br>more popular then current 'priv_size' apporach.<br>But I believe it shouldn't be a build time option.<br>><br>>>> Mbuf dynamic fields are shared across all mbuf pools, and serves the<br>>> need with an existing API. So I am strongly in favor of using the mbuf<br>>> dynamic fields API for this.<br>>>> I agree with Konstantin that it would be optimal if the size of the<br>>> added dynfields area was run-time configurable (as an EAL startup<br>>> parameter).<br>>>> However, such a modification to the mbuf library would also require<br>>> that the performance cost in the dataplane is negligible. We don't want<br>>> to compromise on mbuf performance for applications not using this new<br>>> feature.<br>>>> Randy,<br>>>> Could you please explore such an approach?<br>>>><br>>>> -Morten<o:p></o:p></span></p></div></div></div></div></div></body></html>