<html xmlns:v="urn:schemas-microsoft-com:vml" xmlns:o="urn:schemas-microsoft-com:office:office" xmlns:w="urn:schemas-microsoft-com:office:word" xmlns:m="http://schemas.microsoft.com/office/2004/12/omml" xmlns="http://www.w3.org/TR/REC-html40">
<head>
<meta http-equiv="Content-Type" content="text/html; charset=us-ascii">
<meta name="Generator" content="Microsoft Word 15 (filtered medium)">
<!--[if !mso]><style>v\:* {behavior:url(#default#VML);}
o\:* {behavior:url(#default#VML);}
w\:* {behavior:url(#default#VML);}
.shape {behavior:url(#default#VML);}
</style><![endif]--><style><!--
/* Font Definitions */
@font-face
        {font-family:"Cambria Math";
        panose-1:2 4 5 3 5 4 6 3 2 4;}
@font-face
        {font-family:Calibri;
        panose-1:2 15 5 2 2 2 4 3 2 4;}
@font-face
        {font-family:-apple-system;}
@font-face
        {font-family:"var\(--ff-mono\)";
        panose-1:0 0 0 0 0 0 0 0 0 0;}
@font-face
        {font-family:inherit;
        panose-1:0 0 0 0 0 0 0 0 0 0;}
/* Style Definitions */
p.MsoNormal, li.MsoNormal, div.MsoNormal
        {margin:0in;
        font-size:11.0pt;
        font-family:"Calibri",sans-serif;
        mso-fareast-language:EN-US;}
a:link, span.MsoHyperlink
        {mso-style-priority:99;
        color:#0563C1;
        text-decoration:underline;}
span.EmailStyle17
        {mso-style-type:personal-compose;
        font-family:"Calibri",sans-serif;
        color:windowtext;}
.MsoChpDefault
        {mso-style-type:export-only;
        font-family:"Calibri",sans-serif;
        mso-fareast-language:EN-US;}
@page WordSection1
        {size:8.5in 11.0in;
        margin:1.0in 1.0in 1.0in 1.0in;}
div.WordSection1
        {page:WordSection1;}
/* List Definitions */
@list l0
        {mso-list-id:1744260765;
        mso-list-template-ids:944131308;}
@list l1
        {mso-list-id:1932155144;
        mso-list-template-ids:-1467342282;}
@list l1:level1
        {mso-level-number-format:bullet;
        mso-level-text:\F0B7;
        mso-level-tab-stop:.5in;
        mso-level-number-position:left;
        text-indent:-.25in;
        mso-ansi-font-size:10.0pt;
        font-family:Symbol;}
@list l1:level2
        {mso-level-number-format:bullet;
        mso-level-text:\F0B7;
        mso-level-tab-stop:1.0in;
        mso-level-number-position:left;
        text-indent:-.25in;
        mso-ansi-font-size:10.0pt;
        font-family:Symbol;}
@list l1:level3
        {mso-level-number-format:bullet;
        mso-level-text:\F0B7;
        mso-level-tab-stop:1.5in;
        mso-level-number-position:left;
        text-indent:-.25in;
        mso-ansi-font-size:10.0pt;
        font-family:Symbol;}
@list l1:level4
        {mso-level-number-format:bullet;
        mso-level-text:\F0B7;
        mso-level-tab-stop:2.0in;
        mso-level-number-position:left;
        text-indent:-.25in;
        mso-ansi-font-size:10.0pt;
        font-family:Symbol;}
@list l1:level5
        {mso-level-number-format:bullet;
        mso-level-text:\F0B7;
        mso-level-tab-stop:2.5in;
        mso-level-number-position:left;
        text-indent:-.25in;
        mso-ansi-font-size:10.0pt;
        font-family:Symbol;}
@list l1:level6
        {mso-level-number-format:bullet;
        mso-level-text:\F0B7;
        mso-level-tab-stop:3.0in;
        mso-level-number-position:left;
        text-indent:-.25in;
        mso-ansi-font-size:10.0pt;
        font-family:Symbol;}
@list l1:level7
        {mso-level-number-format:bullet;
        mso-level-text:\F0B7;
        mso-level-tab-stop:3.5in;
        mso-level-number-position:left;
        text-indent:-.25in;
        mso-ansi-font-size:10.0pt;
        font-family:Symbol;}
@list l1:level8
        {mso-level-number-format:bullet;
        mso-level-text:\F0B7;
        mso-level-tab-stop:4.0in;
        mso-level-number-position:left;
        text-indent:-.25in;
        mso-ansi-font-size:10.0pt;
        font-family:Symbol;}
@list l1:level9
        {mso-level-number-format:bullet;
        mso-level-text:\F0B7;
        mso-level-tab-stop:4.5in;
        mso-level-number-position:left;
        text-indent:-.25in;
        mso-ansi-font-size:10.0pt;
        font-family:Symbol;}
ol
        {margin-bottom:0in;}
ul
        {margin-bottom:0in;}
--></style><!--[if gte mso 9]><xml>
<o:shapedefaults v:ext="edit" spidmax="1026" />
</xml><![endif]--><!--[if gte mso 9]><xml>
<o:shapelayout v:ext="edit">
<o:idmap v:ext="edit" data="1" />
</o:shapelayout></xml><![endif]-->
</head>
<body lang="EN-GB" link="#0563C1" vlink="#954F72" style="word-wrap:break-word">
<div class="WordSection1">
<p class="MsoNormal" style="mso-margin-bottom-alt:auto;background:white;vertical-align:baseline">
<span style="font-size:11.5pt;font-family:-apple-system;color:#232629;mso-fareast-language:EN-GB">We have a DPDK application that only calls </span><span style="font-size:10.0pt;font-family:"var(--ff-mono)",serif;color:#232629;border:none windowtext 1.0pt;padding:0in;mso-fareast-language:EN-GB">rte_eth_rx_burst()</span><span style="font-size:11.5pt;font-family:-apple-system;color:#232629;mso-fareast-language:EN-GB"> (we
 do not transmit packets) and it must process the payload very quickly. </span><b><span style="font-size:11.5pt;font-family:"inherit",serif;color:#232629;border:none windowtext 1.0pt;padding:0in;mso-fareast-language:EN-GB">The payload of a single network packet
 MUST be in contiguous memory.</span></b><span style="font-size:11.5pt;font-family:-apple-system;color:#232629;mso-fareast-language:EN-GB"><o:p></o:p></span></p>
<p class="MsoNormal" style="mso-margin-bottom-alt:auto;background:white;vertical-align:baseline">
<span style="font-size:11.5pt;font-family:-apple-system;color:#232629;mso-fareast-language:EN-GB">The DPDK API is optimized around having memory pools of </span><b><span style="font-size:11.5pt;font-family:"inherit",serif;color:#232629;border:none windowtext 1.0pt;padding:0in;mso-fareast-language:EN-GB">fixed-size</span></b><span style="font-size:11.5pt;font-family:-apple-system;color:#232629;mso-fareast-language:EN-GB"> mbufs
 in memory pools. If a packet is received on the DPDK port that is larger than the mbuf size, but smaller than the max MTU then it will be segmented according to the figure in the
<a href="https://doc.dpdk.org/guides/_images/mbuf2.svg">mbuf documentation</a>:<o:p></o:p></span></p>
<p class="MsoNormal" style="mso-margin-bottom-alt:auto;background:white;vertical-align:baseline">
<span style="color:black"><a href="https://i.stack.imgur.com/3mlE4.png"><span style="font-size:11.5pt;font-family:"inherit",serif;color:blue;border:none windowtext 1.0pt;padding:0in;mso-fareast-language:EN-GB;text-decoration:none"><img border="0" width="707" height="348" style="width:7.3645in;height:3.625in" id="Picture_x0020_1" src="cid:image001.png@01D8A1D9.DC86DA30" alt="enter image description here"></span></a></span><span style="font-size:11.5pt;font-family:-apple-system;color:#232629;mso-fareast-language:EN-GB"><o:p></o:p></span></p>
<p class="MsoNormal" style="mso-margin-bottom-alt:auto;background:white;vertical-align:baseline">
<span style="font-size:11.5pt;font-family:-apple-system;color:#232629;mso-fareast-language:EN-GB">This leads us the following problems:<o:p></o:p></span></p>
<p class="MsoNormal" style="margin-left:58.5pt;text-indent:-.25in;mso-list:l1 level1 lfo1;background:white;vertical-align:baseline">
<![if !supportLists]><span style="font-size:10.0pt;font-family:Symbol;color:#232629;mso-fareast-language:EN-GB"><span style="mso-list:Ignore">·<span style="font:7.0pt "Times New Roman"">        
</span></span></span><![endif]><span style="font-size:11.5pt;font-family:"inherit",serif;color:#232629;mso-fareast-language:EN-GB">If we configure the memory pool to store large packets (for example max MTU size) then we will always store the payload in contiguous
 memory, but we will waste huge amounts memory in the case we receive traffic containing small packets. Imagine that our mbuf size is 9216 bytes, but we are receiving mostly packets of size 100-300 bytes. We are wasting memory by a factor of 90!<o:p></o:p></span></p>
<p class="MsoNormal" style="margin-left:58.5pt;text-indent:-.25in;mso-list:l1 level1 lfo1;background:white;vertical-align:baseline">
<![if !supportLists]><span style="font-size:10.0pt;font-family:Symbol;color:#232629;mso-fareast-language:EN-GB"><span style="mso-list:Ignore">·<span style="font:7.0pt "Times New Roman"">        
</span></span></span><![endif]><span style="font-size:11.5pt;font-family:"inherit",serif;color:#232629;mso-fareast-language:EN-GB">If we reduce the size of mbufs, to let's say 512 bytes, then we need special handling of those segments to store the payload in
 contiguous memory. Special handling and copying of segments hurt our performance, so it should be limited.<o:p></o:p></span></p>
<p class="MsoNormal" style="margin-left:58.5pt;background:white;vertical-align:baseline">
<span style="font-size:11.5pt;font-family:"inherit",serif;color:#232629;mso-fareast-language:EN-GB"><o:p> </o:p></span></p>
<p class="MsoNormal" style="mso-margin-bottom-alt:auto;background:white;vertical-align:baseline">
<span style="font-size:11.5pt;font-family:-apple-system;color:#232629;mso-fareast-language:EN-GB">Considering the above,  my questions are as follows:<o:p></o:p></span></p>
<p class="MsoNormal" style="margin-left:58.5pt;text-indent:-.25in;mso-list:l0 level1 lfo2;background:white;vertical-align:baseline">
<![if !supportLists]><span style="font-size:11.5pt;font-family:"inherit",serif;color:#232629;mso-fareast-language:EN-GB"><span style="mso-list:Ignore">1.<span style="font:7.0pt "Times New Roman"">     
</span></span></span><![endif]><span style="font-size:11.5pt;font-family:"inherit",serif;color:#232629;mso-fareast-language:EN-GB">What strategy is recommended for a DPDK application that needs to process the payload of network packets in contiguous memory?
 With <b><span style="border:none windowtext 1.0pt;padding:0in">both</span></b> small (100-300 bytes) and large (9216) packets, without wasting huge amounts of memory with 9K-sized mbuf pools? Is copying segmented jumbo frames into a larger max_mtu mbuf the
 only option?<o:p></o:p></span></p>
<p class="MsoNormal" style="margin-left:58.5pt;text-indent:-.25in;mso-list:l0 level1 lfo2;background:white;vertical-align:baseline">
<![if !supportLists]><span style="font-size:11.5pt;font-family:"inherit",serif;color:#232629;mso-fareast-language:EN-GB"><span style="mso-list:Ignore">2.<span style="font:7.0pt "Times New Roman"">     
</span></span></span><![endif]><span style="font-size:11.5pt;font-family:"inherit",serif;color:#232629;mso-fareast-language:EN-GB">Some frameworks and drivers allow decoupling the RX descriptors (mbuf) from the payload, such that payloads are stored contiguously.
 This does not seem to be possible in the DPDK framework for segmented packets, since mbufs are stored right next to their payload in memory?<o:p></o:p></span></p>
<p class="MsoNormal"><o:p> </o:p></p>
</div>
<hr>
<font size="1" color="#C0C0C0">Pico Quantitative Trading LLC ("PQT"). This e-mail (including any attachments) is intended only for use by the addressee(s) named above, and may contain confidential, proprietary or legally privileged information. If you are not
 the intended recipient of this e-mail, any review, use, disclosure, dissemination, distribution, printing or copying of this e-mail or any attachment is strictly prohibited. If you have received this e-mail in error, please notify Pico immediately by return
 e-mail and permanently delete the original from your system and any hard copy printout thereof. E-mails are not encrypted and cannot be guaranteed to be secure or error-free and, as with all Internet communications, information could be intercepted, corrupted,
 lost, destroyed, arrive late or incomplete, or contain viruses. Accordingly, Pico accepts no liability for any errors or omissions in the content contained herein. In compliance with applicable laws, rules and regulations and/or at its discretion, Pico may
 review and archive incoming and outgoing e-mail communications, copies of which may be produced at the request of regulators.
</font>
</body>
</html>