[dpdk-ci] Minutes of DPDK Lab Meeting, July 17th

[Jeremy Plsek]

Hi Ferruh,

With regards to "DPDK Branch(s) to test", I agree with testing the
patch against the appropriate branch.

For some history, we originally tried to apply patches to the
appropriate branches (before Patchwork 2.0) based on the heuristics of
the subject line, but we ended up having a lot of apply errors. After
a meeting, the group agreed with testing against master for now.

But with Patchwork 2.0, is there a way to tell what branch a
patch/series is supposed to be applied against? If not, how can we
figure this out in a reliable programmatic way?

On Thu, Aug 16, 2018 at 5:33 AM Yigit, Ferruh <ferruh.yigit at intel.com> wrote:
>
> Hi Bob,
>
>
>
> A few comments/questions on “UNH Policies and Procedures Doc”:
>
>
>
> - Objective and Scope of Project
>
>                 - 2: “DPDK-enabled applications”, what are these applications, are they refer to testpmd/l2fwd like test applications, or OVS/VPP like other products using DPDK? And if these are other products, will they run for all vendors or for the vendor requested it?
>
>                 - 3: “Demonstrate any new feature performance of DPDK”, is this via updating test scripts, if not how these new test will be run and how results will be shared?
>
>                 - What is the plan to use Lab for Continuous Integration, adding more sanity checks than performance checks by time, as far as I know this was one of the initial plans for a lab.
>
>
>
> - DPDK Branch(s) to test
>
>                 - 5.1.1: “master”, a little background, in dpdk development there are multiple sub-trees, some specific patches targets specific trees, these sub-trees are merged into main tree before release candidate, and there is a target to do regular integration from these sub-trees.
>
> As a result of this process, for example a patch sent for next-net sub-tree may not apply cleanly to main repo, so it won’t be tested. But that patch will be applied to next-net tree and a week later next-net tree will be merged into main tree, this patch can be something affects the performance, but it won’t be detected. Later, when a patch arrives that can be applied on main tree, it will reveal the performance issue, but suspect will be the wrong patch and the problematic patch will be already merged. We need a solution for this. There are 5 sub-trees merged into main tree and more than half of the patches are coming to main repo through them.
>
>
>
> - Private DPDK-Member only Dashboard Specification
>
> - 5.6.1.2: “The delta-values of the script output, per test performed.”, in member-only dashboard, why not show base value too, since it will be updated regularly, via “--update-expected argument”, it would be good to see both current baseline and the diff.
>
>
>
> - I am for defining a change management system, there are multiple vendor and multiple requests, it would be good to trace, discuss and record the result for all of them systematically.
>
>
>
> Thanks,
>
> ferruh
>
>
>
> From: O'Driscoll, Tim
> Sent: Thursday, July 19, 2018 3:55 PM
> To: 'ci at dpdk.org' <ci at dpdk.org>
> Cc: 'Bob Noseworthy' <ren at iol.unh.edu>; Mcnamara, John <john.mcnamara at intel.com>; 'Shepard Siegel' <shepard.siegel at atomicrules.com>; 'Thomas Monjalon' <thomas at monjalon.net>; 'Erez Scop' <erezsc at mellanox.com>; 'Shreyansh Jain' <shreyansh.jain at nxp.com>; Xu, Qian Q <qian.q.xu at intel.com>; 'pmacarth at iol.unh.edu' <pmacarth at iol.unh.edu>; 'Matt Spencer' <Matt.Spencer at arm.com>; 'George Zhao' <George.Y.Zhao at huawei.com>; 'Mishra, Shishir' <Shishir.Mishra at spirent.com>; 'Lixuming' <lixuming at huawei.com>; Tkachuk, Georgii <georgii.tkachuk at intel.com>; 'Trishan de Lanerolle' <tdelanerolle at linuxfoundation.org>; 'Sean Campbell' <scampbel at qti.qualcomm.com>; 'Ali Alnubani' <alialnu at mellanox.com>; 'May Chen' <May.Chen at huawei.com>; 'Lodha, Nishant' <Nishant.Lodha at cavium.com>; Zhang, Chun <chun.zhang at intel.com>; 'Malla, Malathi' <Malathi.Malla at spirent.com>; 'khemendra kumar' <khemendra.kumar13 at gmail.com>; 'graeme.gregory at linaro.org' <graeme.gregory at linaro.org>; Yigit, Ferruh <ferruh.yigit at intel.com>; Tu, Lijuan <lijuan.tu at intel.com>
> Subject: Minutes of DPDK Lab Meeting, July 17th
>
>
>
> UNH Policies and Procedures Doc:
>
> Document is available at: https://docs.google.com/document/d/1rtSwpNKltVNyDKNWgeTV5gaYeDoAPlK-sfm8XE7o_5s/edit?usp=sharing
>
> I reviewed it and it looks good to me.
>
> Please add any comments to the google doc, or send them directly to Bob (ren at iol.unh.edu).
>
>
>
> Dashboard:
>
> Dashboard is in much better shape now and most tests are passing.
>
> We do need to monitor the results and determine if the tolerance is right. If it’s too high then all tests will pass and we’ll miss any genuine issues. If it’s too low then we’ll have too many tests failing when there isn’t really a problem. Agreed that tuning the tolerance is the responsibility of each vendor.
>
> For Intel, we need to add the specific test config to the results page. We need to confirm the config details to Patrick, and he’ll add this to the results pages. Other vendors may want to do something similar.
>
>
>
> Hardware:
>
> Shreyansh is almost ready to ship the NXP hardware. It should be in UNH in 2-3 weeks.
>
>
>
>
>
>
>
>



--
Jeremy Plsek
UNH InterOperability Laboratory