Email Based Re-Testing Framework

[Patrick Robb]

Hello all,

I'd like to revive the conversation about a request from the community for
an email based re-testing framework. The idea is that using one
standardized format, dpdk developers could email the test-report mailing
list, requesting a rerun on their patch series for "X" set of tests at "Y"
lab. I think that since patchwork testing labels (ie.
iol-broadcom-Performance, github-robot: build, loongarch-compilation) are
already visible on patch pages on patchwork, those labels are the most
reasonable ones to expect developers to use when requesting a re-test. We
probably wouldn't want to get any more general than that, like, say,
rerunning all CI testing for a specific patch series at a specific lab,
since it would result in a significant amount of "wasted" testing capacity.

The standard email format those of us at the Community Lab are thinking of
is like below. Developers would request retests by emailing the test-report
mailing list with email bodies like:

[RETEST UNH-IOL]
iol-abi-testing
iol-broadcom-Performance

[RETEST Intel]
intel-Functional

[RETEST Loongson]
loongarch-compilation

[RETEST GHA]
github-robot: build

>From there, it would be up to the various labs to poll the test-report
mailing list archive (or use a similar method) to check for such requests,
and trigger a CI testing rerun based on the labels provided in the re-test
email. If there is interest from other labs, UNH might also be able to host
the entire set of re-test requests, allowing other labs to poll a curated
list hosted by UNH. One simple approach would be for labs to download all
emails sent to test-report and parse with regex to determine the re-test
list for their specific lab. But, if anyone has any better ideas for
aggregating the emails to be parsed, suggestions are welcome! If this
approach sounds reasonable to everyone, we could determine a timeline by
which labs would implement the functionality needed to trigger re-tests.
Or, we can just add re-testing for various labs if/when they add this
functionality - whatever is better. Happy to discuss at the CI meeting on
Thursday.

-- 

Patrick Robb

Technical Service Manager

UNH InterOperability Laboratory

21 Madbury Rd, Suite 100, Durham, NH 03824

www.iol.unh.edu
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mails.dpdk.org/archives/ci/attachments/20230606/aecefba1/attachment.htm>