W3C home > Mailing lists > Public > public-test-infra@w3.org > January to March 2013

Re: Review of tests upstreamed by implementors

From: Linss, Peter <peter.linss@hp.com>
Date: Wed, 20 Mar 2013 18:58:59 +0000
To: Tobie Langel <tobie@w3.org>
CC: Rebecca Hauck <rhauck@adobe.com>, public-test-infra <public-test-infra@w3.org>
Message-ID: <070F3EA7-4329-44C4-9F32-5E85A31667C7@hp.com>
On Mar 20, 2013, at 10:08 AM, Tobie Langel wrote:

> On Wednesday, March 20, 2013 at 5:52 PM, Rebecca Hauck wrote:
>> This hits on another issue
>> that has also been a bit of a bottleneck which is the requirement* that a
>> review is done by someone outside of your company/organization.
> 
> This isn't actually documented formally anywhere, but I've also heard it mentioned numerous times.
> 
> I proposed that we drop this requirement and explicitly state so in the new review process, i.e. something along the lines of:
> 
> "Contributions must be reviewed by a peer. The reviewer can be a colleague of the contributor as long as the proceedings are public."
> 

Actually IMO this is a case of "what we would like" vs "what actually happens".

FWIW, tests submitted to the CSSWG repository don't need to be reviewed (or approved) before they get included in the build test suites. The build picks up everything not in an "incoming" folder.

Bt the time a spec meets its CR exits criteria, we essentially consider that all the tests have been reviewed at least by the virtue of having been run by multiple people in multiple implementations and no-one complaining about the results. We then effectively approve all the tests in the suite, including those that have not ben through the formal review process. At least this is what we did with CSS2.1. While we aspire to do better and get each test individually reviewed, we have to be pragmatic at the end of the day.

Peter
Received on Wednesday, 20 March 2013 19:00:41 UTC

This archive was generated by hypermail 2.3.1 : Tuesday, 6 January 2015 20:20:33 UTC