RE: Status of reflection tests

>My reflection tests have been sitting around for months.  I've promptly addressed all feedback I received.  

Thanks for addressing all the feedback from the group

>At this point I'd like to ask that either 1) ?they all be approved immediately, or 2) some definite criteria be agreed upon that they have to meet to >be approved.  

The group agreed (see the attached email) that we will push to review all tests that have been submitted to Hg.
The plan was for this to occur from Oct 15th -> Dec 15th - no objections have occurred since.

> It is not acceptable for large test suites with no known problems to languish indefinitely without approval.  We're going to need many millions of >tests to thoroughly cover all of HTML, and we have to have a functional test approval system to handle them.

I think it's really great that you/Google have provided these tests for anyone to use so that the web can be more interoperable.  I do want to set expectations to you and your employer 'Google' that getting consensus that a million tests are correct per the HTML5 spec is going to take some serious time.  That said the test have been submitted to the w3c and can be ran by anyone which has a lot of value even without being 'approved'.  This is similar to a specification that has been published by the editor as a 'draft'.
 
Please note that I believe people do have good intentions in the group.  For example participants from both Opera and Microsoft have helped review and fix the audio/video tests that were submitted by Google.  Even though the tests were never updated by the 'Google' submitter post the test submission after multiple attempts to have him/her update the tests.

So please be patient... 

-Kris
-----Original Message-----
From: public-html-testsuite-request@w3.org [mailto:public-html-testsuite-request@w3.org] On Behalf Of Aryeh Gregor
Sent: Friday, September 23, 2011 12:54 PM
To: public-html-testsuite@w3.org
Subject: Status of reflection tests

My reflection tests have been sitting around for months.  I've promptly addressed all feedback I received.  At this point I'd like to ask that either 1) they all be approved immediately, or 2) some definite criteria be agreed upon that they have to meet to be approved.  I've already offered to make whatever changes people feel are necessary, such as rewriting the test suite in some well-specified way that people feel will make it easier to understand, provided that we agree the tests will be approved after I do it.

It is not acceptable for large test suites with no known problems to languish indefinitely without approval.  We're going to need many millions of tests to thoroughly cover all of HTML, and we have to have a functional test approval system to handle them.