[dpdk-moving] proposal for DPDK CI improvement

[Xu, Qian Q]

Thomas, 
Thx for your quick response. See my reply inline below. 

-----Original Message-----
From: Thomas Monjalon [mailto:thomas.monjalon at 6wind.com] 
Sent: Monday, November 7, 2016 6:18 PM
To: Xu, Qian Q <qian.q.xu at intel.com>
Cc: moving at dpdk.org; Liu, Yong <yong.liu at intel.com>; ci at dpdk.org
Subject: Re: [dpdk-moving] proposal for DPDK CI improvement


> a). Currently, there is only " S/W/F for Success/Warning/Fail counters" in tests, so does it refer to build test or functional test or performance test? 

It can be any test, including performance ones. A major performance regression must be seen as a failed test. 

[Qian] If it refer to any test, so how do we know which test has been done. For example, some patch may only do build, some may do perf and some may do functional. 
How to differentiate these tests execution?   

> If it only referred to build test, then you may need change the title 
> to Build S/W/F. Then how many architecture or platforms for the 
> builds? For example, we support Intel IA build, ARM build, IBM power build. Then we may need collect build results from INTEL/IBM/ARM and etc to show the total S/W/F. For example, if the build is passed on IA but failed on IBM, then we Need record it as 1S/0W/1F. I don't know if we need collect the warning information here.

The difference between warnings and failures is a matter of severity.
The checkpatch errors are reported as warnings.

> b). How about performance result display on website? No matter 
> distributed or centralized lab, we need a place to show the performance number or the performance trend to ensure no performance regression? Do you have any plan to implement it?

No I have no plan but I expect it to be solved by ones working on performance tests, maybe you? :) If a private lab can publish some web graphs of performance evolutions, it is great.
If we can do it in a centralized lab, it is also great.
If we can have a web interface to gather every performance numbers and graphs, it is really really great!

[Qian] In intel internally, we put some efforts on the performance web interface design and implementation. But it is just started. If community need our effort to work on the performance 
Report center, we may need discuss about the ownership/resources/requirements and plan.  

> 3.  Proposal to have a CI mailing list for people working on CI to 
> have the regular meetings only discussing about CI? Maybe we can have more frequent meetings at first to have an alignment. Then We can reduce the frequency if the solution is settle down. Current call is covering many other topics. What do you think?

The mailing list is now created: ci at dpdk.org.
About meetings, I feel we can start working through ci at dpdk.org and see how efficient it is. Though if you need a meeting, feel free to propose.
[Qian] I saw there was a community call tomorrow night, and it was 11pm at PRC time. I wonder if all CI people are in EU, if no US people, then we may wish an early time such as 9pm-10pm at PRC time. 
We can join this week's community call and decide if we can have a separate meeting next week for CI, more focus and more efficient, just for CI.