W3C home > Mailing lists > Public > public-test-infra@w3.org > January to March 2013

FW: Re: Coverage analysis

From: SULLIVAN, BRYAN L <bs3131@att.com>
Date: Thu, 21 Feb 2013 22:07:41 +0000
To: "'Robin Berjon (robin@w3.org)'" <robin@w3.org>
CC: "public-test-infra@w3.org" <public-test-infra@w3.org>
Message-ID: <59A39E87EA9F964A836299497B686C35109BCD56@WABOTH9MSGUSR8D.ITServices.sbc.com>
Hi Robin,

I didn't see a response to the request below. Any clarifications you can add?

Sent: Wednesday, February 13, 2013 9:43 AM
To: Robin Berjon (robin@w3.org); public-test-infra@w3.org
Subject: Re: Coverage analysis

Hi Robin,

Re http://w3c-test.org/html-testsuite/master/tools/coverage/, this is much appreciated and I would like to help expand it to other tests, as well as dig deeper into it for HTML5.

Some requests for clarification:

1.       How was the "tests" column determined? Is this current tests or estimated tests needed?

2.       If the former, do you intent to add a column for "estimated tests needed"?

3.       Why the huge "Tests" # for "parsing"?

4.       I've heard that there are ~11K tests in the HTML WG suite, but there are only approved test 1500 files in the repository. Is there a mapping somewhere between these?

5.       In the testing meeting, I heard (it's in the minutes) of "10000" features in HTML5 and estimates of $100-$200/test with a rollup to $1-$2M in effort to develop them. These numbers don't seem to match up. If there are already ~11K tests, how many tests are expected across all 10000 features of HTML5? The resulting rolled up cost would seem to be much more than $2M unless "features" was meant rather than "tests", re the cost per each.

Bryan Sullivan
Received on Thursday, 21 February 2013 22:08:23 UTC

This archive was generated by hypermail 2.4.0 : Friday, 17 January 2020 17:34:08 UTC