<html> <head> <meta http-equiv="Content-Type" content="text/html; charset=big5"> <style type="text/css" style="display:none;"> P {margin-top:0;margin-bottom:0;} </style> </head> <body dir="ltr"> <div style="font-family: Aptos, Aptos_EmbeddedFont, Aptos_MSFontService, Calibri, Helvetica, sans-serif; font-size: 12pt; color: rgb(0, 0, 0);" class="elementToProof"> Hi all,</div> <div style="margin-top: 1em; margin-bottom: 1em; font-family: Aptos, Aptos_EmbeddedFont, Aptos_MSFontService, Calibri, Helvetica, sans-serif; font-size: 12pt; color: rgb(0, 0, 0);" class="elementToProof"> I have a design question regarding <code>rte_ring</code> that I didn’t find a historical rationale for in the archives.</div> <div style="margin-top: 1em; margin-bottom: 1em; font-family: Aptos, Aptos_EmbeddedFont, Aptos_MSFontService, Calibri, Helvetica, sans-serif; font-size: 12pt; color: rgb(0, 0, 0);" class="elementToProof"> Most modern high-perf ring buffers (e.g. some NIC drivers, some DB queue implementations, etc.) eliminate wrap-around branches by taking the ring’s element array and mapping <i>two</i> consecutive VA ranges to the same physical backing pages.</div> <div style="margin-top: 1em; margin-bottom: 1em; font-family: Aptos, Aptos_EmbeddedFont, Aptos_MSFontService, Calibri, Helvetica, sans-serif; font-size: 12pt; color: rgb(0, 0, 0);" class="elementToProof"> i.e. you allocate N elements, commit enough pages to cover N, then call <code>mmap</code> (or equivalent) again immediately following it, pointing to the same physical pages. So from the CPU’s POV the element array is logically <code>[0 .. N*2)</code> but physically it’s the same backing. Therefore a batch read/write can always be done as a single contiguous memcpy/CLD/STOS without conditionals, even if (head+bulk) exceeds N.</div> <div style="margin-top: 1em; margin-bottom: 1em; font-family: Aptos, Aptos_EmbeddedFont, Aptos_MSFontService, Calibri, Helvetica, sans-serif; font-size: 12pt; color: rgb(0, 0, 0);" class="elementToProof"> Pseudo illustration:</div> <pre class="elementToProof"><div style="font-family: Aptos, Aptos_EmbeddedFont, Aptos_MSFontService, Calibri, Helvetica, sans-serif; font-size: 12pt; color: rgb(0, 0, 0);" class="elementToProof"><code>[phys buffer of N slots] VA: [0 .. N) -> phys VA: [N .. 2N) -> same phys </code></div></pre> <div style="margin-top: 1em; margin-bottom: 1em; font-family: Aptos, Aptos_EmbeddedFont, Aptos_MSFontService, Calibri, Helvetica, sans-serif; font-size: 12pt; color: rgb(0, 0, 0);" class="elementToProof"> For multi-element enqueue/dequeue it eliminates the “if wrap → split” case entirely — you can always memcpy in one contiguous op.</div> <div style="margin-top: 1em; margin-bottom: 1em; font-family: Aptos, Aptos_EmbeddedFont, Aptos_MSFontService, Calibri, Helvetica, sans-serif; font-size: 12pt; color: rgb(0, 0, 0);" class="elementToProof"> <b>Question:</b></div> <div style="margin-top: 1em; margin-bottom: 1em; font-family: Aptos, Aptos_EmbeddedFont, Aptos_MSFontService, Calibri, Helvetica, sans-serif; font-size: 12pt; color: rgb(0, 0, 0);" class="elementToProof"> Is there an explicit reason DPDK doesn’t use this technique for <code>rte_ring</code>?</div> <div style="margin-top: 1em; margin-bottom: 1em; font-family: Aptos, Aptos_EmbeddedFont, Aptos_MSFontService, Calibri, Helvetica, sans-serif; font-size: 12pt; color: rgb(0, 0, 0);" class="elementToProof"> e.g.</div> <ul> <li style="font-family: Aptos, Aptos_EmbeddedFont, Aptos_MSFontService, Calibri, Helvetica, sans-serif; font-size: 12pt; color: rgb(0, 0, 0);"> <div style="margin-top: 1em; margin-bottom: 1em;" role="presentation" class="elementToProof"> portability? (hugepages / VFIO?)</div> </li><li style="font-family: Aptos, Aptos_EmbeddedFont, Aptos_MSFontService, Calibri, Helvetica, sans-serif; font-size: 12pt; color: rgb(0, 0, 0);"> <div style="margin-top: 1em; margin-bottom: 1em;" role="presentation">inability to rely on <code>mmap</code> trickery for hugepage backed memzones?</div> </li><li style="font-family: Aptos, Aptos_EmbeddedFont, Aptos_MSFontService, Calibri, Helvetica, sans-serif; font-size: 12pt; color: rgb(0, 0, 0);"> <div style="margin-top: 1em; margin-bottom: 1em;" role="presentation">NUMA locality considerations?</div> </li><li style="font-family: Aptos, Aptos_EmbeddedFont, Aptos_MSFontService, Calibri, Helvetica, sans-serif; font-size: 12pt; color: rgb(0, 0, 0);"> <div style="margin-top: 1em; margin-bottom: 1em;" role="presentation">historical reason: first gen ring didn’t bulk enqueue so the branch didn’t matter?</div> </li><li style="font-family: Aptos, Aptos_EmbeddedFont, Aptos_MSFontService, Calibri, Helvetica, sans-serif; font-size: 12pt; color: rgb(0, 0, 0);"> <div style="margin-top: 1em; margin-bottom: 1em;" role="presentation">reluctance to add VA aliasing because of security / introspection tooling / ASan issues?</div> </li></ul> <div style="margin-top: 1em; margin-bottom: 1em; font-family: Aptos, Aptos_EmbeddedFont, Aptos_MSFontService, Calibri, Helvetica, sans-serif; font-size: 12pt; color: rgb(0, 0, 0);" class="elementToProof"> I just want to understand the architectural trade that was made.</div> <div style="margin-top: 1em; margin-bottom: 1em; font-family: Aptos, Aptos_EmbeddedFont, Aptos_MSFontService, Calibri, Helvetica, sans-serif; font-size: 12pt; color: rgb(0, 0, 0);" class="elementToProof"> Because on 64-bit Linux, double-mapping a 1–2 MB region is pretty trivial, and bulk ops in DPDK are extremely common — it feels like an obvious microarchitectural win for the “hot ring” case.</div> <div style="margin-top: 1em; margin-bottom: 1em; font-family: Aptos, Aptos_EmbeddedFont, Aptos_MSFontService, Calibri, Helvetica, sans-serif; font-size: 12pt; color: rgb(0, 0, 0);" class="elementToProof"> So: is there a concrete blocker? or simply “no one pushed a patch because current perf was ‘good enough’”?</div> <div style="margin-top: 1em; margin-bottom: 1em; font-family: Aptos, Aptos_EmbeddedFont, Aptos_MSFontService, Calibri, Helvetica, sans-serif; font-size: 12pt; color: rgb(0, 0, 0);" class="elementToProof"> Pointers to prior mailing list discussion / patches would be appreciated.</div> <div style="margin-top: 1em; margin-bottom: 1em; font-family: Aptos, Aptos_EmbeddedFont, Aptos_MSFontService, Calibri, Helvetica, sans-serif; font-size: 12pt; color: rgb(0, 0, 0);" class="elementToProof"> Thanks,<br> Rami Neiman</div> <div style="font-family: Aptos, Aptos_EmbeddedFont, Aptos_MSFontService, Calibri, Helvetica, sans-serif; font-size: 12pt; color: rgb(0, 0, 0);" class="elementToProof"> <br> </div> <hr> Confidentiality notice <br> <br> This e-mail message and any attachment hereto contain confidential information which may be privileged and which is intended for the exclusive use of its addressee(s). If you receive this message in error, please inform sender immediately and destroy any copy thereof. Furthermore, any disclosure, distribution or copying of this message and/or any attachment hereto without the consent of the sender is strictly prohibited. Thank you. </body> </html>