- From: Arthur Barstow <art.barstow@nokia.com>
- Date: Mon, 08 Apr 2013 10:20:33 -0400
- To: "ext Zhao, Tina" <tina.zhao@intel.com>
- CC: "public-webapps-testsuite@w3.org" <public-webapps-testsuite@w3.org>
On 4/3/13 3:17 AM, ext Zhao, Tina wrote: > > The draft the Opera's SSE test report is available at > http://www.w3.org/wiki/Webapps/Interop/ServerSentEvents > > I have a question about test result measurement: > > Take [1] test file for example. There are four check points in one > file, only if all of them get pass, the test result shows PASS (other > ways Fail) in report, right? However, in traditional QA way, I thought > this should be four test cases, 3 PASS, 1 NotRun (so I highlighted the > details in report). > > Do we have rule or guideline for this? > No, we don't have any strict rules regarding test result reporting. If a row/line in the test report is a summary of more than one test, then if any of that row's tests fail, I think the Principle of Least Surprise suggests an overall status of Fail makes sense. It is also acceptable to report the results such that every test has its own row/line (even if there are multiple tests per file). Presumably that would be more work for you, but AFAIC, that would be fine. (This is what Lachlan did with the Selectors API Implementation Report <http://dev.w3.org/2006/webapi/selectors-api-testsuite/level1-baseline-report.html>.) -Art > [1] > https://github.com/w3c/event-source/blob/master/eventsource-constructor-stringify.htm > > Thanks, > > Tina >
Received on Monday, 8 April 2013 14:21:06 UTC