Re: Test coverage

On Wednesday, March 27, 2013 at 11:46 PM, Rebecca Hauck wrote:
> I'm somewhat of the mind that something is better than nothing. If a spec
> doesn't have a person to make these assessments, we can only look at the
> statistics that the spec analysis algorithm gives us. It's not ideal, but
> better than completely guessing.

 Oh, absolutely.
> BTW, I just closed my previous pull request and replaced it with a new one
> [1] - json data is there for all of the specs in TV and Mobile profiles,
> with the exception of CSS2.1.

Awesome. That PR looks very similar to the previous one. Am I missing anything?
> For some reason that spec isn't being
> parsed. I'm debugging it right now, but didn't want it to hold up the rest.

 It's multi-page!
> 
> -Rebecca
> 
> [1] https://github.com/w3c/web-platform-tests/pull/43
> 
> On 3/27/13 3:25 PM, "Tobie Langel" <tobie@w3.org (mailto:tobie@w3.org)> wrote:
> 
> > Right, I'm trying to validate the spec-analysis model. Hence trying to
> > get out of this circular logic.
> > 
> > --tobie 
> > 
> > 
> > On Wednesday, March 27, 2013 at 11:23 PM, Rebecca Hauck wrote:
> > 
> > > If you mean the way they have Status: XXX% complete, no, not that I'm
> > > aware of. I don't think I've ever seen any estimates for "how many tests
> > > do we need to be complete?" that would inform that percentage number.
> > > The
> > > best we have to go on is the spec analysis + shepherd data for now.
> > > 
> > > 
> > > 
> > > On 3/27/13 3:10 PM, "Tobie Langel" <tobie@w3.org (mailto:tobie@w3.org)>
> > > wrote:
> > > 
> > > > I was curious whether there had been any manual assessment of test
> > > > coverage. Eg. how that had been done in WebApps[1].
> > > > 
> > > > --tobie
> > > > 
> > > > ---
> > > > [1]: http://www.w3.org/2008/webapps/wiki/PubStatus
> > > > 
> > > > 
> > > > On Wednesday, March 27, 2013 at 10:57 PM, Rebecca Hauck wrote:
> > > > 
> > > > > The landing page of Shepherd summarizes the test counts for each
> > > spec:
> > > > > 
> > > > > http://test.csswg.org/shepherd/
> > > > > 
> > > > > 
> > > > > You can drill down from there.
> > > > > 
> > > > > With the shepherd API in the coverage scripts, we'll see the same
> > > > > numbers
> > > > > aligned with their respective spec sections with the word analysis.
> > > > > 
> > > > > Note: If you don't see the spec you're looking for on the Shepherd
> > > main
> > > > > page, that means there are no tests.
> > > > > 
> > > > > 
> > > > > On 3/27/13 1:22 PM, "Tobie Langel" <tobie@w3.org (mailto:tobie@w3.org)
> > > (mailto:tobie@w3.org)>
> > > > > wrote:
> > > > > 
> > > > > > Hi, 
> > > > > > 
> > > > > > Do we have any info on the level of coverage of the different test
> > > > > suites?
> > > > > > 
> > > > > > Thanks,
> > > > > > 
> > > > > > --tobie

Received on Wednesday, 27 March 2013 22:50:53 UTC