W3C home > Mailing lists > Public > public-html-testsuite@w3.org > August 2010

Re: XHTML5 Test Submission

From: James Graham <jgraham@opera.com>
Date: Tue, 24 Aug 2010 12:48:10 +0200
Message-ID: <4C73A36A.3040302@opera.com>
To: Kris Krueger <krisk@microsoft.com>
CC: "'public-html-testsuite@w3.org' (public-html-testsuite@w3.org)" <public-html-testsuite@w3.org>
On 06/24/2010 01:25 AM, Kris Krueger wrote:
> In an effort to improve browser interoperability, Microsoft has submitted more HTML5 tests for XHTML5.
> Microsoft welcomes feedback on the tests if they are incorrect per the HTML5 specification.
>
> http://test.w3.org/html/tests/submission/Microsoft/

(Apologies I missed this email earlier).

These tests are all manual tests. However they are typically not testing 
things that *require* manual testing. Given the high costs of running 
manual tests and the relative difficulty of adding them to automated 
regression testing systems, I believe this should be sufficient grounds 
to reject the tests. Also, it is important for Process reasons that 
compiling implementation reports is simple. Having even a small 
percentage of manual tests out of the tens of thousands of tests that 
HTML5 will require will constitute a significant burden on the 
production and update of implementation reports. Therefore I propose 
that we have a policy that manual tests be accompanied by an 
explaination of why the test _must_ be manual and cannot be implemented 
as either a javascript test or a reftest.
Received on Tuesday, 24 August 2010 10:48:51 GMT

This archive was generated by hypermail 2.2.0+W3C-0.50 : Tuesday, 24 August 2010 10:48:51 GMT