W3C home > Mailing lists > Public > public-test-infra@w3.org > January to March 2013

Re: Coverage analysis

From: SULLIVAN, BRYAN L <bs3131@att.com>
Date: Wed, 13 Feb 2013 15:43:28 +0000
To: "Robin Berjon (robin@w3.org)" <robin@w3.org>, "public-test-infra@w3.org" <public-test-infra@w3.org>
Message-ID: <59A39E87EA9F964A836299497B686C35109914FE@WABOTH9MSGUSR8D.ITServices.sbc.com>
Hi Robin,

Re http://w3c-test.org/html-testsuite/master/tools/coverage/, this is much appreciated and I would like to help expand it to other tests, as well as dig deeper into it for HTML5.

Some requests for clarification:

1.       How was the "tests" column determined? Is this current tests or estimated tests needed?

2.       If the former, do you intent to add a column for "estimated tests needed"?

3.       Why the huge "Tests" # for "parsing"?

4.       I've heard that there are ~11K tests in the HTML WG suite, but there are only approved test 1500 files in the repository. Is there a mapping somewhere between these?

5.       In the testing meeting, I heard (it's in the minutes) of "10000" features in HTML5 and estimates of $100-$200/test with a rollup to $1-$2M in effort to develop them. These numbers don't seem to match up. If there are already ~11K tests, how many tests are expected across all 10000 features of HTML5? The resulting rolled up cost would seem to be much more than $2M unless "features" was meant rather than "tests", re the cost per each.

Bryan Sullivan
Received on Wednesday, 13 February 2013 15:44:20 UTC

This archive was generated by hypermail 2.3.1 : Tuesday, 6 January 2015 20:20:33 UTC