W3C home > Mailing lists > Public > public-test-infra@w3.org > January to March 2013

Re: Review of tests upstreamed by implementors

From: Arthur Barstow <art.barstow@nokia.com>
Date: Thu, 21 Mar 2013 14:50:25 -0400
Message-ID: <514B5671.9090509@nokia.com>
To: Tobie Langel <tobie@w3.org>
CC: public-test-infra <public-test-infra@w3.org>
On 3/20/13 5:05 AM, ext Tobie Langel wrote:
> One of the topic that came up on multiple occasions was that tests upstreamed by implementers had already been peer-reviewed internally.
>
> It seems giving special treatment to such submissions would help reduce the bottleneck and get tests in the repository much faster. (Note that we could still run a number of tests automatically on such submission to catch common issues).
>
> In order to go through this fast-track process, some form of log of the internal review process would need to be produced alongside the submission. For open-source projects, this could be an URL to a publicly accessible bug tracker, for non open-source projects, this would need to be added to the body of the pull request.
>
> Should a given submission prove problematic, the merge would be reverted and the tests would go through the regular review process.
>
> Thoughts?

The process I am familiar with is that a WG is responsible for approving 
the tests used to determine if a spec meets in CR exit criteria.

Would you please clarify if you are proposing the WG defer to the 
upstream review (and not do its own review and approval); or more like 
this additional bit of data would be given to the WG when they 
review+approve tests for a CR; or something else?

(I'm sure the overall workflow your proposal builds upon is documented 
somewhere so I apologize for the dumb question but I couldn't find that 
document so please do let me know where the "new" workflow is documented.)

-Thanks, AB
Received on Thursday, 21 March 2013 18:50:47 UTC

This archive was generated by hypermail 2.3.1 : Tuesday, 6 January 2015 20:20:33 UTC