W3C home > Mailing lists > Public > public-html-testsuite@w3.org > March 2013

Re: Coverage analysis

From: Tobie Langel <tobie@w3.org>
Date: Wed, 6 Mar 2013 19:05:15 +0100
To: Rebecca Hauck <rhauck@adobe.com>
Cc: Linss, Peter <peter.linss@hp.com>, Robin Berjon <robin@w3.org>, "public-test-infra@w3.org" <public-test-infra@w3.org>, "public-html-testsuite@w3.org" <public-html-testsuite@w3.org>, "public-css-testsuite@w3.org" <public-css-testsuite@w3.org>
Message-ID: <377F0DA5475B4259B7439D820D28E5AB@w3.org>
On Tuesday, February 26, 2013 at 8:31 PM, Rebecca Hauck wrote:
> I generated some reports for a couple of CSS specs (Transforms and
> Backgrounds & Borders) and used some of the CSS infrastructure. I still
> used Robin's spec parser, but pulled the number of tests from our existing
> DB (from the annotations on the specs).
> 
> A few changes to Robin's scripts to support this:
> 
> - (minor) Parse the ToC whether it's an ol or a ul - CSS specs use the
> latter; HTML the former
> - Bypass the test-per-section.json and pull the test counts directly from
> the spec if it has these annotations.
> 
> Both methods can peacefully coexist now, but we may want to discuss
> converging them (like if the other Wgs wanted to adopt annotate.js).
> 
> I sent a pull request with these changes:
> https://github.com/w3c/html-testsuite/pull/33
> 
> In this pull request, I've added a separate css-index.html for the CSS
> reports with the accompanying json files. I realize the html-testsuite
> repo is probably not the logical home for these, but perhaps we can
> discuss a more centralized place for all things coverage in the Testing
> Task Force.

Agreed. The goal is to rename the html-testsuite repository and include all OWP test suites in it.
> I had trouble building gh-pages on my fork of the entire html-testsuite,
> so I temporarily have the CSS reports here until the pull request is
> merged or they're moved elsewhere:
> 
> http://rhauck.github.com/css-coverage/
> 
> There are a few flaws in these reports (for example, the Backgrounds &
> Borders report probably doesn't need to include changes, acknowledgements,
> and indices). I'll write up a list of ideas and things I noticed as I dug
> deeper into this. Again, perhaps a good thing to address in the Task Force.

Sounds like we should be improving our heuristics for CSS to include all the propdef blocks. 
> 
> Robin, thanks again for getting this going! It was well documented and
> straightforward to adapt, which is always appreciated. Now that we have
> something functioning, it should be easy tweak & refine it to make this
> more useful over time. I'm happy to help move it forward!

Great work on this. Thanks for sharing.

Worked on the UI a bit to offer a more opinionated view which is thus much easier to parse:

http://tobie.github.com/css-coverage/index.html

I'm planning to generalize and automate this process for all specs we'll be testing in the near future.

--tobie 
Received on Wednesday, 6 March 2013 18:05:34 GMT

This archive was generated by hypermail 2.2.0+W3C-0.50 : Wednesday, 6 March 2013 18:05:34 GMT