FW: Re: Coverage analysis

Hi Robin,

I didn't see a response to the request below. Any clarifications you can add?

From: SULLIVAN, BRYAN L
Sent: Wednesday, February 13, 2013 9:43 AM
To: Robin Berjon (robin@w3.org); public-test-infra@w3.org
Subject: Re: Coverage analysis

Hi Robin,

Re http://w3c-test.org/html-testsuite/master/tools/coverage/, this is much appreciated and I would like to help expand it to other tests, as well as dig deeper into it for HTML5.

Some requests for clarification:


1.       How was the "tests" column determined? Is this current tests or estimated tests needed?


2.       If the former, do you intent to add a column for "estimated tests needed"?


3.       Why the huge "Tests" # for "parsing"?


4.       I've heard that there are ~11K tests in the HTML WG suite, but there are only approved test 1500 files in the repository. Is there a mapping somewhere between these?


5.       In the testing meeting, I heard (it's in the minutes) of "10000" features in HTML5 and estimates of $100-$200/test with a rollup to $1-$2M in effort to develop them. These numbers don't seem to match up. If there are already ~11K tests, how many tests are expected across all 10000 features of HTML5? The resulting rolled up cost would seem to be much more than $2M unless "features" was meant rather than "tests", re the cost per each.

Thanks,
Bryan Sullivan