<html xmlns:v="urn:schemas-microsoft-com:vml" xmlns:o="urn:schemas-microsoft-com:office:office" xmlns:w="urn:schemas-microsoft-com:office:word" xmlns:dt="uuid:C2F41010-65B3-11d1-A29F-00AA00C14882" xmlns:m="http://schemas.microsoft.com/office/2004/12/omml" xmlns="http://www.w3.org/TR/REC-html40">
<head>
<meta http-equiv="Content-Type" content="text/html; charset=us-ascii">
<meta name="Generator" content="Microsoft Word 15 (filtered medium)">
<style><!--
/* Font Definitions */
@font-face
{font-family:"Cambria Math";
panose-1:2 4 5 3 5 4 6 3 2 4;}
@font-face
{font-family:Calibri;
panose-1:2 15 5 2 2 2 4 3 2 4;}
@font-face
{font-family:"Malgun Gothic";
panose-1:2 11 5 3 2 0 0 2 0 4;}
@font-face
{font-family:"\@Malgun Gothic";}
@font-face
{font-family:D2Coding;}
/* Style Definitions */
p.MsoNormal, li.MsoNormal, div.MsoNormal
{margin:0cm;
text-align:justify;
text-autospace:none;
word-break:break-all;
font-size:10.0pt;
font-family:"Malgun Gothic",sans-serif;}
a:link, span.MsoHyperlink
{mso-style-priority:99;
color:#0563C1;
text-decoration:underline;}
span.EmailStyle19
{mso-style-type:personal-reply;
font-family:"Calibri",sans-serif;
color:windowtext;}
.MsoChpDefault
{mso-style-type:export-only;
font-size:10.0pt;}
@page WordSection1
{size:612.0pt 792.0pt;
margin:3.0cm 72.0pt 72.0pt 72.0pt;}
div.WordSection1
{page:WordSection1;}
--></style><!--[if gte mso 9]><xml>
<o:shapedefaults v:ext="edit" spidmax="1026" />
</xml><![endif]--><!--[if gte mso 9]><xml>
<o:shapelayout v:ext="edit">
<o:idmap v:ext="edit" data="1" />
</o:shapelayout></xml><![endif]-->
</head>
<body lang="EN-US" link="#0563C1" vlink="#954F72" style="word-wrap:break-word">
<div class="WordSection1">
<p class="MsoNormal" style="word-break:normal"><span style="font-size:11.0pt;font-family:"Calibri",sans-serif">Hi Jaeeun,<o:p></o:p></span></p>
<p class="MsoNormal" style="word-break:normal"><span style="font-size:11.0pt;font-family:"Calibri",sans-serif">See my responses inline below.<o:p></o:p></span></p>
<p class="MsoNormal" style="word-break:normal"><span style="font-size:11.0pt;font-family:"Calibri",sans-serif"><o:p> </o:p></span></p>
<p class="MsoNormal" style="word-break:normal"><span style="font-size:11.0pt;font-family:"Calibri",sans-serif">-Jay<o:p></o:p></span></p>
<p class="MsoNormal" style="word-break:normal"><span style="font-size:11.0pt;font-family:"Calibri",sans-serif"><o:p> </o:p></span></p>
<p class="MsoNormal" style="word-break:normal"><span style="font-size:11.0pt;font-family:"Calibri",sans-serif"><o:p> </o:p></span></p>
<div style="border:none;border-left:solid blue 1.5pt;padding:0cm 0cm 0cm 4.0pt">
<div>
<div style="border:none;border-top:solid #E1E1E1 1.0pt;padding:3.0pt 0cm 0cm 0cm">
<p class="MsoNormal" align="left" style="text-align:left;text-autospace:ideograph-numeric ideograph-other;word-break:normal">
<b><span style="font-size:11.0pt;font-family:"Calibri",sans-serif">From:</span></b><span style="font-size:11.0pt;font-family:"Calibri",sans-serif"> Jaeeun Ham <jaeeun.ham@ericsson.com>
<br>
<b>Sent:</b> Monday, June 13, 2022 5:51 AM<br>
<b>To:</b> dev@dpdk.org<br>
<b>Cc:</b> Jerin Jacob <jerinj@marvell.com>; Jayatheerthan, Jay <jay.jayatheerthan@intel.com><br>
<b>Subject:</b> ask for TXA_FLUSH_THRESHOLD change<o:p></o:p></span></p>
</div>
</div>
<p class="MsoNormal" align="left" style="text-align:left"><o:p> </o:p></p>
<p class="MsoNormal"><span style="font-family:"Arial",sans-serif;mso-fareast-language:KO">Hi,<o:p></o:p></span></p>
<p class="MsoNormal"><span style="font-family:"Arial",sans-serif;mso-fareast-language:KO"><o:p> </o:p></span></p>
<p class="MsoNormal"><span style="font-family:"Arial",sans-serif;mso-fareast-language:KO">There were latency delay when I increase dpdk(20.11.1) core. (one worker core was okay.)<o:p></o:p></span></p>
<p class="MsoNormal"><span style="font-family:"Arial",sans-serif;mso-fareast-language:KO">When I decrease the TXA_FLUSH_THRESHOLD value(1024 to 32), it was okay
<o:p></o:p></span></p>
<p class="MsoNormal"><span style="font-family:"Arial",sans-serif;mso-fareast-language:KO"><o:p> </o:p></span></p>
<p class="MsoNormal"><span style="font-family:"Arial",sans-serif;mso-fareast-language:KO">It’s TXA_FLUSH_THRESHOLD at lib/librte_eventdev/rte_event_eth_tx_adapter.c. //
<a href="https://git.dpdk.org/dpdk-stable/tree/lib/librte_eventdev/rte_event_eth_tx_adapter.c?h=20.11#n15">
https://git.dpdk.org/dpdk-stable/tree/lib/librte_eventdev/rte_event_eth_tx_adapter.c?h=20.11#n15</a><o:p></o:p></span></p>
<p class="MsoNormal"><span style="font-family:"Arial",sans-serif;mso-fareast-language:KO">When TXA_FLUSH_THRESHOLD value was changed from
<b>1024</b> to <b>32</b>, the latency test result was fine on 10 cores for low traffic(DL:20Mbps/UL:17kbps).<o:p></o:p></span></p>
<p class="MsoNormal"><span style="font-family:"Arial",sans-serif;mso-fareast-language:KO">I think that this can make call for rte_eth_tx_buffer_flush() more frequently.<o:p></o:p></span></p>
<p class="MsoNormal"><b><span style="font-family:"Arial",sans-serif;mso-fareast-language:KO">But, I’m not sure whether this approach can cause worse performance or not.<o:p></o:p></span></b></p>
<p class="MsoNormal"><span style="font-family:"Arial",sans-serif;mso-fareast-language:KO">Do you have any opinion about this?<o:p></o:p></span></p>
<p class="MsoNormal"><span style="font-size:11.0pt;font-family:"Calibri",sans-serif;mso-fareast-language:KO"><o:p> </o:p></span></p>
<p class="MsoNormal"><span style="font-size:11.0pt;font-family:"Calibri",sans-serif;mso-fareast-language:KO">[Jay] Yes, it will cause rte_eth_tx_buffer_flush() to be called more often. It can lead to lesser batching benefit. Typical performance vs. latency
trade-off decision apply here.<o:p></o:p></span></p>
<p class="MsoNormal"><span style="font-size:11.0pt;font-family:"Calibri",sans-serif;mso-fareast-language:KO"><o:p> </o:p></span></p>
<p class="MsoNormal"><span style="font-family:"Arial",sans-serif;mso-fareast-language:KO"><o:p> </o:p></span></p>
<p class="MsoNormal"><span style="font-family:"Arial",sans-serif;mso-fareast-language:KO">Similar RDK RTE_BRIDGE_ETH_TX_FLUSH_THOLD is patched on DUSG3 from 1024 to smaller value since DPDK 18.11.2:<o:p></o:p></span></p>
<p class="MsoNormal"><span style="font-family:"Arial",sans-serif;mso-fareast-language:KO">I’m not aware of any side-effect, I think it is needed to have low enough latency even at low traffic rates. For more details see Intel
<a href="https://footprints.intel.com/MRcgi/MRlogin.pl?DL=22288DA14">FP 22288</a>.<o:p></o:p></span></p>
<p class="MsoNormal"><span style="font-size:11.0pt;font-family:"Calibri",sans-serif;mso-fareast-language:KO"><o:p> </o:p></span></p>
<p class="MsoNormal"><span style="font-size:11.0pt;font-family:"Calibri",sans-serif;mso-fareast-language:KO">[Jay] Currently,
</span><span style="font-family:"Arial",sans-serif;mso-fareast-language:KO">TXA_FLUSH_THRESHOLD</span><span style="font-size:11.0pt;font-family:"Calibri",sans-serif;mso-fareast-language:KO"> is not a configurable attr.<o:p></o:p></span></p>
<p class="MsoNormal"><span style="font-family:"Arial",sans-serif;mso-fareast-language:KO"><o:p> </o:p></span></p>
<p class="MsoNormal"><span style="font-family:"Arial",sans-serif;mso-fareast-language:KO">TXA_MAX_NB_TX(<b>128</b>) looks the same as CONFIG_RTE_BRIDGE_ETH_EVENT_BUFFER_SIZE(<b>16384</b>), then
<b>is it also should be tuned?<o:p></o:p></b></span></p>
<p class="MsoNormal"><span style="font-size:11.0pt;font-family:"Calibri",sans-serif;mso-fareast-language:KO"><o:p> </o:p></span></p>
<p class="MsoNormal"><span style="font-size:11.0pt;font-family:"Calibri",sans-serif;mso-fareast-language:KO">[Jay] They both are different attributes.
</span><span style="font-family:"Arial",sans-serif;mso-fareast-language:KO">TXA_MAX_NB_TX refers to max number of queues in Tx adapter. CONFIG_RTE_BRIDGE_ETH_EVENT_BUFFER_SIZE refers to event buffer size in Rx BD.</span><span style="font-size:11.0pt;font-family:"Calibri",sans-serif;mso-fareast-language:KO"><o:p></o:p></span></p>
<p class="MsoNormal"><span style="font-family:"Arial",sans-serif;mso-fareast-language:KO"><o:p> </o:p></span></p>
<p class="MsoNormal"><span style="font-family:"Arial",sans-serif;mso-fareast-language:KO"><o:p> </o:p></span></p>
<p class="MsoNormal" style="margin-left:10.0pt;mso-para-margin-left:1.0gd"><i><span style="font-family:D2Coding;mso-fareast-language:KO">--- dpdk-3pp-swu-18.11/dpdk-stable-18.11.2/config/common_base.orig 2020-01-29 15:05:10.000000000 +0100<o:p></o:p></span></i></p>
<p class="MsoNormal" style="margin-left:10.0pt;mso-para-margin-left:1.0gd"><i><span style="font-family:D2Coding;mso-fareast-language:KO">+++ dpdk-3pp-swu-18.11/dpdk-stable-18.11.2/config/common_base 2020-01-29 15:11:10.000000000 +0100<o:p></o:p></span></i></p>
<p class="MsoNormal" style="margin-left:10.0pt;mso-para-margin-left:1.0gd"><i><span style="font-family:D2Coding;mso-fareast-language:KO">@@ -566,9 +566,9 @@<o:p></o:p></span></i></p>
<p class="MsoNormal" style="margin-left:10.0pt;mso-para-margin-left:1.0gd"><i><span style="font-family:D2Coding;mso-fareast-language:KO">CONFIG_RTE_LIBRTE_BRIDGE_ETH_MAX_CP_ENQ_RETRIES=100<o:p></o:p></span></i></p>
<p class="MsoNormal" style="margin-left:10.0pt;mso-para-margin-left:1.0gd"><i><span style="font-family:D2Coding;mso-fareast-language:KO">CONFIG_RTE_MAX_BRIDGE_ETH_INSTANCE=4<o:p></o:p></span></i></p>
<p class="MsoNormal" style="margin-left:10.0pt;mso-para-margin-left:1.0gd"><i><span style="font-family:D2Coding;mso-fareast-language:KO">CONFIG_RTE_BRIDGE_ETH_INTR_RING_SIZE=32<o:p></o:p></span></i></p>
<p class="MsoNormal" style="margin-left:10.0pt;mso-para-margin-left:1.0gd"><i><span style="font-family:D2Coding;color:red;mso-fareast-language:KO">-CONFIG_RTE_BRIDGE_ETH_EVENT_BUFFER_SIZE=128<o:p></o:p></span></i></p>
<p class="MsoNormal" style="margin-left:10.0pt;mso-para-margin-left:1.0gd"><i><span style="font-family:D2Coding;color:#4472C4;mso-fareast-language:KO">+CONFIG_RTE_BRIDGE_ETH_EVENT_BUFFER_SIZE=16384<o:p></o:p></span></i></p>
<p class="MsoNormal" style="margin-left:10.0pt;mso-para-margin-left:1.0gd"><i><span style="font-family:D2Coding;mso-fareast-language:KO">CONFIG_RTE_LIBRTE_BRIDGE_ETH_DEBUG=n<o:p></o:p></span></i></p>
<p class="MsoNormal" style="margin-left:10.0pt;mso-para-margin-left:1.0gd"><i><span style="font-family:D2Coding;color:red;mso-fareast-language:KO">-CONFIG_RTE_BRIDGE_ETH_TX_FLUSH_THOLD=1024<o:p></o:p></span></i></p>
<p class="MsoNormal" style="margin-left:10.0pt;mso-para-margin-left:1.0gd"><i><span style="font-family:D2Coding;color:#4472C4;mso-fareast-language:KO">+CONFIG_RTE_BRIDGE_ETH_TX_FLUSH_THOLD=<b>32</b><o:p></o:p></span></i></p>
<p class="MsoNormal" style="margin-left:10.0pt;mso-para-margin-left:1.0gd"><i><span style="font-family:D2Coding;mso-fareast-language:KO"><o:p> </o:p></span></i></p>
<p class="MsoNormal" style="margin-left:10.0pt;mso-para-margin-left:1.0gd"><i><span style="font-family:D2Coding;mso-fareast-language:KO">--- dpdk-3pp-swu-dusg3-20.11.3/dpdk-stable-20.11.3/config/rte_config.h 2021-08-05 23:46:52.051051000 +0200<o:p></o:p></span></i></p>
<p class="MsoNormal" style="margin-left:10.0pt;mso-para-margin-left:1.0gd"><i><span style="font-family:D2Coding;mso-fareast-language:KO">+++ dpdk-3pp-swu-dusg3-20.11.3/dpdk-stable-20.11.3/config/rte_config.h 2021-08-06 00:50:07.310766255 +0200<o:p></o:p></span></i></p>
<p class="MsoNormal" style="margin-left:10.0pt;mso-para-margin-left:1.0gd"><i><span style="font-family:D2Coding;mso-fareast-language:KO">@@ -175,8 +175,8 @@<o:p></o:p></span></i></p>
<p class="MsoNormal" style="margin-left:10.0pt;mso-para-margin-left:1.0gd"><i><span style="font-family:D2Coding;mso-fareast-language:KO">#define RTE_LIBRTE_BRIDGE_ETH_MAX_CP_ENQ_RETRIES 100<o:p></o:p></span></i></p>
<p class="MsoNormal" style="margin-left:10.0pt;mso-para-margin-left:1.0gd"><i><span style="font-family:D2Coding;mso-fareast-language:KO">#define RTE_MAX_BRIDGE_ETH_INSTANCE 4<o:p></o:p></span></i></p>
<p class="MsoNormal" style="margin-left:10.0pt;mso-para-margin-left:1.0gd"><i><span style="font-family:D2Coding;mso-fareast-language:KO">#define RTE_BRIDGE_ETH_INTR_RING_SIZE 32<o:p></o:p></span></i></p>
<p class="MsoNormal" style="margin-left:10.0pt;mso-para-margin-left:1.0gd"><i><span style="font-family:D2Coding;color:red;mso-fareast-language:KO">-#define RTE_BRIDGE_ETH_EVENT_BUFFER_SIZE 128<o:p></o:p></span></i></p>
<p class="MsoNormal" style="margin-left:10.0pt;mso-para-margin-left:1.0gd"><i><span style="font-family:D2Coding;color:red;mso-fareast-language:KO">-#define RTE_BRIDGE_ETH_TX_FLUSH_THOLD 1024<o:p></o:p></span></i></p>
<p class="MsoNormal" style="margin-left:10.0pt;mso-para-margin-left:1.0gd"><i><span style="font-family:D2Coding;color:#4472C4;mso-fareast-language:KO">+#define RTE_BRIDGE_ETH_EVENT_BUFFER_SIZE 16384<o:p></o:p></span></i></p>
<p class="MsoNormal" style="margin-left:10.0pt;mso-para-margin-left:1.0gd"><i><span style="font-family:D2Coding;color:#4472C4;mso-fareast-language:KO">+#define RTE_BRIDGE_ETH_TX_FLUSH_THOLD
<b>10</b><o:p></o:p></span></i></p>
<p class="MsoNormal" style="margin-left:10.0pt;mso-para-margin-left:1.0gd"><i><span style="font-family:D2Coding;mso-fareast-language:KO">#undef RTE_BRIDGE_ETH_TX_MULTI_PKT_EVENT<o:p></o:p></span></i></p>
<p class="MsoNormal"><span style="font-family:"Arial",sans-serif;mso-fareast-language:KO"><o:p> </o:p></span></p>
<p class="MsoNormal"><span style="font-family:"Arial",sans-serif;mso-fareast-language:KO"><o:p> </o:p></span></p>
<p class="MsoNormal"><span style="font-family:"Arial",sans-serif;mso-fareast-language:KO">BR/Jaeeun<o:p></o:p></span></p>
<p class="MsoNormal"><span style="mso-fareast-language:KO"><o:p> </o:p></span></p>
</div>
</div>
</body>
</html>