Re: Test coverage

I'm somewhat of the mind that something is better than nothing. If a spec
doesn't have a person to make these assessments, we can only look at the
statistics that the spec analysis algorithm gives us. It's not ideal, but
better than completely guessing.

BTW, I just closed my previous pull request and replaced it with a new one
[1] - json data is there for all of the specs in TV and Mobile profiles,
with the exception of CSS2.1.  For some reason that spec isn't being
parsed. I'm debugging it right now, but didn't want it to hold up the rest.


-Rebecca

[1] https://github.com/w3c/web-platform-tests/pull/43

On 3/27/13 3:25 PM, "Tobie Langel" <tobie@w3.org> wrote:

>Right, I'm trying to validate the spec-analysis model. Hence trying to
>get out of this circular logic.
>
>--tobie 
>
>
>On Wednesday, March 27, 2013 at 11:23 PM, Rebecca Hauck wrote:
>
>> If you mean the way they have Status: XXX% complete, no, not that I'm
>> aware of. I don't think I've ever seen any estimates for "how many tests
>> do we need to be complete?" that would inform that percentage number.
>>The
>> best we have to go on is the spec analysis + shepherd data for now.
>> 
>> 
>> 
>> On 3/27/13 3:10 PM, "Tobie Langel" <tobie@w3.org (mailto:tobie@w3.org)>
>>wrote:
>> 
>> > I was curious whether there had been any manual assessment of test
>> > coverage. Eg. how that had been done in WebApps[1].
>> > 
>> > --tobie
>> > 
>> > ---
>> > [1]: http://www.w3.org/2008/webapps/wiki/PubStatus
>> > 
>> > 
>> > On Wednesday, March 27, 2013 at 10:57 PM, Rebecca Hauck wrote:
>> > 
>> > > The landing page of Shepherd summarizes the test counts for each
>>spec:
>> > > 
>> > > http://test.csswg.org/shepherd/
>> > > 
>> > > 
>> > > You can drill down from there.
>> > > 
>> > > With the shepherd API in the coverage scripts, we'll see the same
>> > > numbers
>> > > aligned with their respective spec sections with the word analysis.
>> > > 
>> > > Note: If you don't see the spec you're looking for on the Shepherd
>>main
>> > > page, that means there are no tests.
>> > > 
>> > > 
>> > > On 3/27/13 1:22 PM, "Tobie Langel" <tobie@w3.org
>>(mailto:tobie@w3.org)>
>> > > wrote:
>> > > 
>> > > > Hi, 
>> > > > 
>> > > > Do we have any info on the level of coverage of the different test
>> > > suites?
>> > > > 
>> > > > Thanks,
>> > > > 
>> > > > --tobie 
>
>

Received on Wednesday, 27 March 2013 22:45:25 UTC