W3C home > Mailing lists > Public > public-html-testsuite@w3.org > August 2010

Re: XHTML5 Test Submission

From: Philippe Le Hegaret <plh@w3.org>
Date: Tue, 24 Aug 2010 11:12:28 -0400
To: James Graham <jgraham@opera.com>
Cc: Kris Krueger <krisk@microsoft.com>, "'public-html-testsuite@w3.org' (public-html-testsuite@w3.org)" <public-html-testsuite@w3.org>
Message-ID: <1282662748.2270.12.camel@chacal>
On Tue, 2010-08-24 at 12:48 +0200, James Graham wrote:
> On 06/24/2010 01:25 AM, Kris Krueger wrote:
> > In an effort to improve browser interoperability, Microsoft has submitted more HTML5 tests for XHTML5.
> > Microsoft welcomes feedback on the tests if they are incorrect per the HTML5 specification.
> >
> > http://test.w3.org/html/tests/submission/Microsoft/
> 
> (Apologies I missed this email earlier).
> 
> These tests are all manual tests. However they are typically not testing 
> things that *require* manual testing. Given the high costs of running 
> manual tests and the relative difficulty of adding them to automated 
> regression testing systems, I believe this should be sufficient grounds 
> to reject the tests. Also, it is important for Process reasons that 
> compiling implementation reports is simple. Having even a small 
> percentage of manual tests out of the tens of thousands of tests that 
> HTML5 will require will constitute a significant burden on the 
> production and update of implementation reports. Therefore I propose 
> that we have a policy that manual tests be accompanied by an 
> explaination of why the test _must_ be manual and cannot be implemented 
> as either a javascript test or a reftest.

I would agree with that but we have no way to run reftests automatically
at the moment. Our test harness doesn't support it.

So, how would we use those reftests?


Philippe
Received on Tuesday, 24 August 2010 15:12:31 GMT

This archive was generated by hypermail 2.2.0+W3C-0.50 : Tuesday, 24 August 2010 15:12:32 GMT