<html> <head> <meta http-equiv="Content-Type" content="text/html; charset=utf-8"> </head> <body> <div style="direction: ltr; font-family: Aptos, Arial, Helvetica, sans-serif; font-size: 12pt; color: rgb(0, 0, 0);"> Morten,</div> <div style="direction: ltr; font-family: Aptos, Arial, Helvetica, sans-serif; font-size: 12pt; color: rgb(0, 0, 0);"> Thanks, this helps clarify the intended direction.</div> <div style="direction: ltr; font-family: Aptos, Arial, Helvetica, sans-serif; font-size: 12pt; color: rgb(0, 0, 0);"> </div> <div style="font-family: Aptos, Arial, Helvetica, sans-serif; font-size: 12pt; color: rgb(0, 0, 0);"> My concern with a dynfield3-style area was not the mechanics of accessing the</div> <div style="font-family: Aptos, Arial, Helvetica, sans-serif; font-size: 12pt; color: rgb(0, 0, 0);"> field. It was whether a comparatively large application-defined mbuf area fit</div> <div style="font-family: Aptos, Arial, Helvetica, sans-serif; font-size: 12pt; color: rgb(0, 0, 0);"> the intended policy for the existing dynfield support.</div> <div style="direction: ltr; font-family: Aptos, Arial, Helvetica, sans-serif; font-size: 12pt; color: rgb(0, 0, 0);"> </div> <div style="font-family: Aptos, Arial, Helvetica, sans-serif; font-size: 12pt; color: rgb(0, 0, 0);"> Our requirement is for a contiguous per-mbuf metadata area that is cache-line</div> <div style="font-family: Aptos, Arial, Helvetica, sans-serif; font-size: 12pt; color: rgb(0, 0, 0);"> aligned. Private data satisfied that shape directly, which is why we started</div> <div style="font-family: Aptos, Arial, Helvetica, sans-serif; font-size: 12pt; color: rgb(0, 0, 0);"> there. If you're agreeable to extending the existing dynfield support this</div> <div style="font-family: Aptos, Arial, Helvetica, sans-serif; font-size: 12pt; color: rgb(0, 0, 0);"> way, that would be a significantly smaller and cleaner approach forward.</div> <div style="direction: ltr; font-family: Aptos, Arial, Helvetica, sans-serif; font-size: 12pt; color: rgb(0, 0, 0);"> </div> <div style="font-family: Aptos, Arial, Helvetica, sans-serif; font-size: 12pt; color: rgb(0, 0, 0);"> The size does need to be configurable. We use the same DPDK package across</div> <div style="font-family: Aptos, Arial, Helvetica, sans-serif; font-size: 12pt; color: rgb(0, 0, 0);"> multiple platforms, including both 32-bit and 64-bit applications, and the</div> <div style="font-family: Aptos, Arial, Helvetica, sans-serif; font-size: 12pt; color: rgb(0, 0, 0);"> per-mbuf size requirement is not necessarily the same across those builds. A</div> <div style="font-family: Aptos, Arial, Helvetica, sans-serif; font-size: 12pt; color: rgb(0, 0, 0);"> fixed expansion would force every consumer to pay the same memory cost.</div> <div style="direction: ltr; font-family: Aptos, Arial, Helvetica, sans-serif; font-size: 12pt; color: rgb(0, 0, 0);"> </div> <div style="font-family: Aptos, Arial, Helvetica, sans-serif; font-size: 12pt; color: rgb(0, 0, 0);"> This also has a useful implementation benefit: if the extra area is accounted</div> <div style="font-family: Aptos, Arial, Helvetica, sans-serif; font-size: 12pt; color: rgb(0, 0, 0);"> for centrally in the mbuf layout/init path, then PMD-managed pools outside the</div> <div style="font-family: Aptos, Arial, Helvetica, sans-serif; font-size: 12pt; color: rgb(0, 0, 0);"> normal pktmbuf create API do not need separate private-size fixups. Existing</div> <div style="font-family: Aptos, Arial, Helvetica, sans-serif; font-size: 12pt; color: rgb(0, 0, 0);"> per-pool private data can remain separate and continue to serve pool-specific</div> <div style="font-family: Aptos, Arial, Helvetica, sans-serif; font-size: 12pt; color: rgb(0, 0, 0);"> users.</div> <div style="direction: ltr; font-family: Aptos, Arial, Helvetica, sans-serif; font-size: 12pt; color: rgb(0, 0, 0);"> </div> <div style="font-family: Aptos, Arial, Helvetica, sans-serif; font-size: 12pt; color: rgb(0, 0, 0);"> With that framing, I am comfortable pursuing this direction.</div> <div style="font-family: Aptos, Arial, Helvetica, sans-serif; font-size: 12pt; color: rgb(0, 0, 0);"> Thank you very much!</div> <div style="font-family: Aptos, Arial, Helvetica, sans-serif; font-size: 12pt; color: rgb(0, 0, 0);"> Regards,</div> <div style="direction: ltr; font-family: Aptos, Arial, Helvetica, sans-serif; font-size: 12pt; color: rgb(0, 0, 0);"> -rt</div> <div style="direction: ltr; font-family: Aptos, Arial, Helvetica, sans-serif; font-size: 12pt; color: rgb(0, 0, 0);"> </div> <div id="mail-editor-reference-message-container"> <div style="padding: 3pt 0in 0in; border-width: 1pt medium medium; border-style: solid none none; border-color: rgb(181, 196, 223) currentcolor currentcolor;"> <div style="text-align: left; font-family: Aptos; font-size: 12pt; color: black;"> From: Morten Brørup <mb@smartsharesystems.com> Date: Tuesday, September 1, 2026 at 12:36 PM To: Randy Tice (rtice) <rtice@cisco.com>; dev@dpdk.org <dev@dpdk.org>; Stephen Hemminger <stephen@networkplumber.org>; Konstantin Ananyev <konstantin.ananyev@huawei.com> Subject: RE: [RFC] mbuf: add configurable base private size for pktmbuf pools </div> </div> <div id="mail-editor-reference-message-body"> <div class="ms-outlook-mobile-reference-message skipProofing" style="direction: ltr;"> <meta name="Generator" content="Microsoft Word 12 (filtered medium)"> </div> <a name="_MailEndCompose" data-outlook-id="9deb4516-b1a1-485e-8684-3f4408a90018" style="margin-top: 0px; margin-bottom: 0px;">Randy,</a> Did you look at the Dynamic Mbuf Fields API? You can reserve a large chunk as an mbuf dynamic field. It will be contiguous. You can also specify alignment when allocating a field. Maybe the size of the dynfield3 area should be run-time configurable; I addressed that option at the end of my original reply. Unless there’s some subtle point I’m misunderstanding, I’m confident Dynamic Mbuf Fields is the right solution for your use case. As the mbuf maintainer, I would welcome such an addition to the Dynamic Mbuf Fields library. Venlig hilsen / Kind regards, -Morten Brørup <div style="padding: 0cm 0cm 0cm 4pt; border-width: medium medium medium 1.5pt; border-style: none none none solid; border-color: currentcolor currentcolor currentcolor blue;"> <div style="padding: 3pt 0cm 0cm; border-width: 1pt medium medium; border-style: solid none none; border-color: rgb(181, 196, 223) currentcolor currentcolor;"> From: Morten Brørup Sent: Tuesday, 1 September 2026 18.23 To: 'Randy Tice (rtice)'; dev@dpdk.org; Stephen Hemminger; Konstantin Ananyev Subject: RE: [RFC] mbuf: add configurable base private size for pktmbuf pools </div> Randy, I don’t understand why you don’t like my proposed solution. What is the difference between adding 384 bytes to all mbufs as an extra dynfield3, compared to adding 384 bytes to all mbufs through application_base_priv? The extra 384 bytes will be located at the same position relative to the mbuf (between the rte_mbuf structure and the priv_data area), and will use the same amount of memory for all mbufs. You propose splitting the priv_data area into two parts: One globally sized area for all mbuf pools (configured through the EAL option), and one per-mempool sized area, configured at pktmbuf-pool creation. Then the global priv_data part does exactly the same the Dynamic Mbuf Fields does. There is no reason to introduce another method for doing something an existing API already does. It’s better to expand the implementation of the existing API. Venlig hilsen / Kind regards, -Morten Brørup <div style="padding: 0cm 0cm 0cm 4pt; border-width: medium medium medium 1.5pt; border-style: none none none solid; border-color: currentcolor currentcolor currentcolor blue;"> <div style="padding: 3pt 0cm 0cm; border-width: 1pt medium medium; border-style: solid none none; border-color: rgb(181, 196, 223) currentcolor currentcolor;"> From: Randy Tice (rtice) [mailto:rtice@cisco.com] Sent: Tuesday, 1 September 2026 17.44 To: Morten Brørup; dev@dpdk.org; Stephen Hemminger; Konstantin Ananyev Subject: Re: [RFC] mbuf: add configurable base private size for pktmbuf pools </div> Morten, Thanks for the detailed response. I went back through our requirements and the current mbuf layout, and I believe our use case maps better to private data. The metadata area we need today is roughly 384 bytes per mbuf. It needs to be contiguous, cache-aligned, and available consistently across the packet mbuf pools used by the platform. The existing reserved areas in struct rte_mbuf do not fit that shape. dynfield1[9] is only 36 bytes and is not cache-line aligned as a standalone area. dynfield2 is only 8 bytes and is configuration-dependent: it exists only when RTE_IOVA_IN_MBUF is 0, meaning the IOVA value is not carried in the mbuf. When RTE_IOVA_IN_MBUF is 1, that mbuf slot is used for other layout state and dynfield2 is absent. So even using those fields directly, they are not large enough or consistent enough for this metadata overlay. We could add a larger reserved area such as the proposed dynfield3, but it would have to be size-adjustable. Not every application would want another fixed 384 bytes in every mbuf, and our own requirement has already moved from 256 to 384 bytes for 64-bit applications, so the size may continue to evolve. That makes a fixed mbuf-struct expansion a poor fit. The existing private data area already exists for this kind of use case: contiguous per-mbuf storage adjacent to the mbuf object, with sizing handled as part of mbuf pool creation. The issue we are trying to solve is that priv_size is currently pool-specific, while our common packet metadata needs a base area available across all packet mbuf pools. The proposed patch adds that common base private size through the standard packet mbuf pool creation helpers. Pool-specific private data can still be added on top when needed. So, for this use case, private data is the cleaner and more direct fit. Thanks, -rt <div id="mail-editor-reference-message-container"> <div style="padding: 3pt 0cm 0cm; border-width: 1pt medium medium; border-style: solid none none; border-color: currentcolor;"> From: Morten Brørup <mb@smartsharesystems.com> Date: Tuesday, September 1, 2026 at 7:11 AM To: Randy Tice (rtice) <rtice@cisco.com>; dev@dpdk.org <dev@dpdk.org>; Stephen Hemminger <stephen@networkplumber.org>; Konstantin Ananyev <konstantin.ananyev@huawei.com> Subject: RE: [RFC] mbuf: add configurable base private size for pktmbuf pools </div> <div id="mail-editor-reference-message-body"> > From: Randy Tice (rtice) [<a href="mailto:rtice@cisco.com" data-outlook-id="7ff1eedb-f5a6-4bbd-a6a7-ceee902d985d" style="color: blue; margin-top: 0px; margin-bottom: 0px;">mailto:rtice@cisco.com</a>] > Sent: Monday, 31 August 2026 20.20 > > Hi, > I would like to get feedback on a proposed mbuf change before sending patches. > Some deployments need a guaranteed private-data reservation in every packet mbuf, across multiple mbuf pools and across different consumers of the mbuf APIs. > Today, each pktmbuf pool can request a private size when the pool is created. That works when the application owns all pool creation policy directly. However, not all relevant mbuf pools are necessarily created by application code. Some pools may be created by libraries, drivers, or other components outside direct application control. > One example already in DPDK is vhost crypto, which creates its own mbuf pool and supplies a private size for struct vhost_crypto_data_req. There are also driver-created pktmbuf-style pools, such as cnxk inline meta pools and TAP GSO context pools. These are examples of pool-creation paths where the application may not directly control the private-size value used at creation time. > A PMD-specific devarg could solve one instance of this problem, such as a single driver-created pool, but that seems too narrow if the requirement is not inherently PMD-specific. A deployment with multiple drivers, libraries, or other pool-creation paths outside application control could need the same base private-size adjustment. In that case, configuring the same value independently through component-specific options would be fragile and easy to get wrong. > The proposed generic model is to add a configurable base private size for pktmbuf pools. The effective private size would be: > align(pool_requested_priv_size + application_base_priv_size, > RTE_MBUF_PRIV_ALIGN) > The tentative EAL option name is: > --mbuf-base-priv-size=<size> > The intent is: > * default behavior remains unchanged when the option is not used; > * the configured base size is added to the private size requested by each pktmbuf pool; > * the final effective private size remains aligned to RTE_MBUF_PRIV_ALIGN; > * pool-specific private-data requests still work as they do today; > * DPDK centralizes the policy so pools created outside application control can reserve the same base private-data space as application-created pools. > This is not intended to define ownership or layout of the private area. It only ensures that a deployment can reserve a common base amount of private data consistently. Applications, drivers, libraries, or components would still be responsible for their own interpretation of the reserved private area. > Questions for the list: > 1. Is a deployment-wide pktmbuf base private-size reservation something DPDK would consider acceptable? > 2. Is --mbuf-base-priv-size=<size> a reasonable name, or would another name better describe the intent? > 3. Should DPDK expose the effective-size calculation as a helper so pool-creation paths outside application control can apply the same rule? > 4. Would maintainers prefer consumer updates in the same series as example users, or as follow-up patches after the generic mbuf/EAL change is accepted? > 5. Would maintainers prefer this to remain component-specific, even if more than one driver, library, or pool-creation path may need to apply the same base reservation? > The main goal is to avoid downstream mbuf layout changes and avoid component-specific configuration drift, while still allowing deployments to reserve a consistent private-data area across all packet mbuf pools, including pools created outside direct application control. > Thanks, > Randy DPDK already has Dynamic Mbuf Fields for run-time management of private data across all mbuf pools. DPDK also has the Private Data Area (priv_size), but that is individual to each mbuf pool, which does not fit your use case. Dynamic Mbuf Fields is the perfect fit for the use case you are describing. Currently, it only manages a few small memory areas inside the rte_mbuf structure itself, the dynfield1 array and the dynfield2 field. But it could easily manage one more memory area associated with the rte_mbuf structure. If we want this to be build-time configurable, it should be relatively simple to add: In config/rte_common.h: +#define RTE_MBUF_DYN_EXTRA_SIZE 128 In lib/mbuf/rte_mbuf_core.h: /** Size of the application private data. In case of an indirect * mbuf, it stores the direct mbuf private data size. */ uint16_t priv_size; /** Timesync flags for use with IEEE1588. */ uint16_t timesync; uint32_t dynfield1[9]; /**< Reserved for dynamic fields. */ + +#if RTE_MBUF_DYN_EXTRA_SIZE + /** Extra dynamic fields. */ + uint32_t dynfield3[RTE_MBUF_DYN_EXTRA_SIZE / sizeof(uint32_t)]; +#endif }; In lib/mbuf/rte_mbuf_dyn.h: +static_assert(RTE_MBUF_DYN_EXTRA_SIZE % RTE_CACHE_LINE_SIZE == 0, + "RTE_MBUF_DYN_EXTRA_SIZE must be multiple of cache line size."); And some associated additions in lib/mbuf/rte_mbuf_dyn.c. <feature creep> There may also be considerations about what happens to the extra data when: - Copying an mbuf. - Attaching an mbuf to another mbuf. - Detaching an mbuf from another mbuf. - Cloning an mbuf. (The considerations apply to both the packet mbuf itself, and for the non-first segments of a segmented packet mbuf.) The developer of the Dynamic Mbuf Fields library was foreseeable enough to add a "flags" parameter for dynamic mbuf field creation. This could be used to specify what happens to each registered dynamic field in the events enumerated above. </feature creep> IMO, the extended size of Dynamic Mbuf Fields should be build-time configurable. If the community wants the extended size of Dynamic Mbuf Fields run-time configurable, the size should be an EAL startup parameter. It could be named: --mbuf-dyn-extra-size=<size>. The major difference in implementation is that the space for the extra dynfields must be dynamically allocated with the mbufs at mbuf pool creation. Notice that the memory for the extra dynfields should be positioned between the mbuf structure and the private data area, so their offsets remain the same, also for two mbuf pools having different Private Data Area sizes. -Morten </div> </div> </div> </div> </div> </div> </body> </html>