<div dir="ltr"><div>Hi Lijuan,<br></div><div><br></div><div>Just a quick update.<br><br></div><div>We had tested the patch on all our bare-metal machines. </div><div><br>And they all work as expected.<br><br>Best,<br>David</div><div><br></div><div><br></div></div><br><div class="gmail_quote"><div dir="ltr" class="gmail_attr">On Thu, Mar 25, 2021 at 11:34 PM Tu, Lijuan <<a href="mailto:lijuan.tu@intel.com">lijuan.tu@intel.com</a>> wrote:<br></div><blockquote class="gmail_quote" style="margin:0px 0px 0px 0.8ex;border-left:1px solid rgb(204,204,204);padding-left:1ex"> <div lang="EN-US" style="overflow-wrap: break-word;"> <div class="gmail-m_-5273405645485565528WordSection1"> <p class="MsoNormal">Hi Owen,<u></u><u></u></p> <p class="MsoNormal"><u></u> <u></u></p> <p class="MsoNormal">I ‘d like to add 2 more changes in V2:<u></u><u></u></p> <p class="MsoNormal"><u></u> <u></u></p> <p class="MsoNormal" style="margin-bottom:5pt">1, test_interval will be renamed to throughput_stat_sample_interval for better understanding.<u></u><u></u></p> <p class="MsoNormal" style="margin-bottom:5pt">2, accepted_tolerance will be a float to algin with delta in json, both format are {:.2f} <u></u><u></u></p> <p class="MsoNormal"><u></u> <u></u></p> <p class="MsoNormal"><u></u> <u></u></p> <div style="border-top:none;border-right:none;border-bottom:none;border-left:1.5pt solid blue;padding:0cm 0cm 0cm 4pt"> <div> <div style="border-right:none;border-bottom:none;border-left:none;border-top:1pt solid rgb(225,225,225);padding:3pt 0cm 0cm"> <p class="MsoNormal"><b>From:</b> dts <<a href="mailto:dts-bounces@dpdk.org" target="_blank">dts-bounces@dpdk.org</a>> <b>On Behalf Of </b> Tu, Lijuan<br> <b>Sent:</b> 2021<span lang="ZH-CN" style="font-family:"Microsoft YaHei",sans-serif">年</span>3<span lang="ZH-CN" style="font-family:"Microsoft YaHei",sans-serif">月</span>25<span lang="ZH-CN" style="font-family:"Microsoft YaHei",sans-serif">日</span> 15:29<br> <b>To:</b> Owen Hilyard <<a href="mailto:ohilyard@iol.unh.edu" target="_blank">ohilyard@iol.unh.edu</a>><br> <b>Cc:</b> David Liu <<a href="mailto:dliu@iol.unh.edu" target="_blank">dliu@iol.unh.edu</a>>; Ali Alnubani <<a href="mailto:alialnu@nvidia.com" target="_blank">alialnu@nvidia.com</a>>; <a href="mailto:dts@dpdk.org" target="_blank">dts@dpdk.org</a>; <a href="mailto:dpdklab@iol.unh.edu" target="_blank">dpdklab@iol.unh.edu</a><br> <b>Subject:</b> Re: [dts] [dpdklab] Re: [PATCH] tests/nic_single_core_perf: modify accepted_tolerance to percentage<u></u><u></u></p> </div> </div> <p class="MsoNormal"><u></u> <u></u></p> <p class="MsoNormal">sure, I prefer to use a float too.<u></u><u></u></p> <p class="MsoNormal"><u></u> <u></u></p> <div style="border-top:none;border-right:none;border-bottom:none;border-left:1.5pt solid blue;padding:0cm 0cm 0cm 4pt"> <div> <div style="border-right:none;border-bottom:none;border-left:none;border-top:1pt solid rgb(225,225,225);padding:3pt 0cm 0cm"> <p class="MsoNormal"><b>From:</b> Owen Hilyard <<a href="mailto:ohilyard@iol.unh.edu" target="_blank">ohilyard@iol.unh.edu</a>> <br> <b>Sent:</b> 2021<span lang="ZH-CN" style="font-family:"Microsoft YaHei",sans-serif">年</span>3<span lang="ZH-CN" style="font-family:"Microsoft YaHei",sans-serif">月</span>24<span lang="ZH-CN" style="font-family:"Microsoft YaHei",sans-serif">日</span> 1:24<br> <b>To:</b> Tu, Lijuan <<a href="mailto:lijuan.tu@intel.com" target="_blank">lijuan.tu@intel.com</a>><br> <b>Cc:</b> David Liu <<a href="mailto:dliu@iol.unh.edu" target="_blank">dliu@iol.unh.edu</a>>; Ali Alnubani <<a href="mailto:alialnu@nvidia.com" target="_blank">alialnu@nvidia.com</a>>; <a href="mailto:dts@dpdk.org" target="_blank">dts@dpdk.org</a>; <a href="mailto:dpdklab@iol.unh.edu" target="_blank"> dpdklab@iol.unh.edu</a><br> <b>Subject:</b> Re: [dpdklab] Re: [dts] [PATCH] tests/nic_single_core_perf: modify accepted_tolerance to percentage<u></u><u></u></p> </div> </div> <p class="MsoNormal"><u></u> <u></u></p> <div> <p class="MsoNormal">Hello Lijuan,<u></u><u></u></p> <div> <p class="MsoNormal"><u></u> <u></u></p> </div> <div> <p class="MsoNormal">Would it be possible to have the delta be a float in the json representation? For example, a 20% delta would be encoded as "delta": .2 in json. Currently, most of the community lab's infrastructure expects to make pass/fail determinations on a numeric type, so having the delta as a float would reduce the required downtime to change everything over to percentages since we shouldn't need to patch all of our services to expect a string. <u></u><u></u></p> <p class="MsoNormal"><u></u> <u></u></p> </div> <div> <p class="MsoNormal"><u></u> <u></u></p> </div> <div> <p class="MsoNormal">Thanks,<u></u><u></u></p> </div> <div> <p class="MsoNormal">Owen Hilyard<u></u><u></u></p> </div> </div> <p class="MsoNormal"><u></u> <u></u></p> <div> <div> <p class="MsoNormal">On Tue, Mar 23, 2021 at 10:53 AM David Liu <<a href="mailto:dliu@iol.unh.edu" target="_blank">dliu@iol.unh.edu</a>> wrote:<u></u><u></u></p> </div> <blockquote style="border-top:none;border-right:none;border-bottom:none;border-left:1pt solid rgb(204,204,204);padding:0cm 0cm 0cm 6pt;margin:5pt 0cm 5pt 4.8pt"> <div> <p class="MsoNormal">Thank you for Ali and Lijuan.<br> <br> We had tested the patch on the Arm machines setup. Other than the value issues, the test is running as expected.<u></u><u></u></p> </div> <p class="MsoNormal"><u></u> <u></u></p> <div> <div> <p class="MsoNormal">On Tue, Mar 23, 2021 at 4:48 AM Tu, Lijuan <<a href="mailto:lijuan.tu@intel.com" target="_blank">lijuan.tu@intel.com</a>> wrote:<u></u><u></u></p> </div> <blockquote style="border-top:none;border-right:none;border-bottom:none;border-left:1pt solid rgb(204,204,204);padding:0cm 0cm 0cm 6pt;margin:5pt 0cm 5pt 4.8pt"> <p class="MsoNormal"><br> <br> > -----Original Message-----<br> > From: Ali Alnubani <<a href="mailto:alialnu@nvidia.com" target="_blank">alialnu@nvidia.com</a>><br> > Sent: 2021<span lang="ZH-CN" style="font-family:DengXian">年</span>3<span lang="ZH-CN" style="font-family:DengXian">月</span>23<span lang="ZH-CN" style="font-family:DengXian">日</span> 16:44<br> > To: Tu, Lijuan <<a href="mailto:lijuan.tu@intel.com" target="_blank">lijuan.tu@intel.com</a>><br> > Cc: <a href="mailto:dts@dpdk.org" target="_blank">dts@dpdk.org</a>; <a href="mailto:dpdklab@iol.unh.edu" target="_blank"> dpdklab@iol.unh.edu</a><br> > Subject: RE: [dts] [PATCH] tests/nic_single_core_perf: modify<br> > accepted_tolerance to percentage<br> > <br> > > -----Original Message-----<br> > > From: dts <<a href="mailto:dts-bounces@dpdk.org" target="_blank">dts-bounces@dpdk.org</a>> On Behalf Of Lijuan Tu<br> > > Sent: Thursday, March 11, 2021 12:48 PM<br> > > To: <a href="mailto:dpdklab@iol.unh.edu" target="_blank">dpdklab@iol.unh.edu</a><br> > > Cc: <a href="mailto:dts@dpdk.org" target="_blank">dts@dpdk.org</a>; Lijuan Tu <<a href="mailto:lijuan.tu@intel.com" target="_blank">lijuan.tu@intel.com</a>><br> > > Subject: [dts] [PATCH] tests/nic_single_core_perf: modify<br> > > accepted_tolerance to percentage<br> > ><br> > > * accepted_tolerance in conf/nic_single_core_perf.cfg are changed to<br> > > percentage, e.g.<br> > > accepted_tolerance = '1%%'<br> > > if not set, it will be set to 2% by default.<br> > > * delat in json are chagned to percentage too, e.g.<br> > > 'delta': '1.5%'<br> > > * expected_throughput will be forced to 1Mpps for compute delat in<br> > > percentage<br> > > * add new conf test_interval in conf/nic_single_core_perf.cfg, which<br> > > indicates interval of getting throughput statistics in second, e.g.<br> > > test_interval = 10<br> > > if not set, it will be set to 5 seconds by default.<br> > > * collect better statistics , get multiple throughput statistics<br> > > during the test run, and compute the average as the final results.<br> > ><br> > > Signed-off-by: Lijuan Tu <<a href="mailto:lijuan.tu@intel.com" target="_blank">lijuan.tu@intel.com</a>><br> > > ---<br> > <snipped><br> > > +accepted_tolerance = '1%%'<br> > <snipped><br> > > + diff =<br> > > + (self.throughput[fwd_config][frame_size][nb_desc] -<br> > > +<br> > > self.expected_throughput[fwd_config][frame_size][nb_desc])/ \<br> > > +<br> > > self.expected_throughput[fwd_config][frame_size][nb_desc]<br> > <br> > The value of `diff` here is still a decimal, and needs to be multiplied by a 100<br> > before it can be compared to `accepted_tolerance`.<br> > <br> > I suggest splitting this patch into multiple ones. One for making the percentage<br> > change and one for taking the mean for multiple readings since they are<br> > unrelated changes.<br> > I also see other unrelated changes not explained in the commit message.<br> <br> Really appreciate for your review, I will accept your advices.<br> <br> > <br> > Thanks,<br> > Ali<u></u><u></u></p> </blockquote> </div> </blockquote> </div> </div> </div> </div> </div> </blockquote></div>