- From: James Graham <jgraham@opera.com>
- Date: Tue, 24 Aug 2010 12:48:10 +0200
- To: Kris Krueger <krisk@microsoft.com>
- CC: "'public-html-testsuite@w3.org' (public-html-testsuite@w3.org)" <public-html-testsuite@w3.org>
On 06/24/2010 01:25 AM, Kris Krueger wrote: > In an effort to improve browser interoperability, Microsoft has submitted more HTML5 tests for XHTML5. > Microsoft welcomes feedback on the tests if they are incorrect per the HTML5 specification. > > http://test.w3.org/html/tests/submission/Microsoft/ (Apologies I missed this email earlier). These tests are all manual tests. However they are typically not testing things that *require* manual testing. Given the high costs of running manual tests and the relative difficulty of adding them to automated regression testing systems, I believe this should be sufficient grounds to reject the tests. Also, it is important for Process reasons that compiling implementation reports is simple. Having even a small percentage of manual tests out of the tens of thousands of tests that HTML5 will require will constitute a significant burden on the production and update of implementation reports. Therefore I propose that we have a policy that manual tests be accompanied by an explaination of why the test _must_ be manual and cannot be implemented as either a javascript test or a reftest.
Received on Tuesday, 24 August 2010 10:48:51 UTC