Re: XHTML5 Test Submission

On Tue, 2010-08-24 at 12:48 +0200, James Graham wrote:
> On 06/24/2010 01:25 AM, Kris Krueger wrote:
> > In an effort to improve browser interoperability, Microsoft has submitted more HTML5 tests for XHTML5.
> > Microsoft welcomes feedback on the tests if they are incorrect per the HTML5 specification.
> >
> > http://test.w3.org/html/tests/submission/Microsoft/
> 
> (Apologies I missed this email earlier).
> 
> These tests are all manual tests. However they are typically not testing 
> things that *require* manual testing. Given the high costs of running 
> manual tests and the relative difficulty of adding them to automated 
> regression testing systems, I believe this should be sufficient grounds 
> to reject the tests. Also, it is important for Process reasons that 
> compiling implementation reports is simple. Having even a small 
> percentage of manual tests out of the tens of thousands of tests that 
> HTML5 will require will constitute a significant burden on the 
> production and update of implementation reports. Therefore I propose 
> that we have a policy that manual tests be accompanied by an 
> explaination of why the test _must_ be manual and cannot be implemented 
> as either a javascript test or a reftest.

I would agree with that but we have no way to run reftests automatically
at the moment. Our test harness doesn't support it.

So, how would we use those reftests?


Philippe

Received on Tuesday, 24 August 2010 15:12:31 UTC