W3C home > Mailing lists > Public > public-css-testsuite@w3.org > March 2013

Re: Shepherd API for number of tests

From: Tobie Langel <tobie@w3.org>
Date: Thu, 28 Mar 2013 00:37:16 +0100
To: "Gérard Talbot" <css21testsuite@gtalbot.org>
Cc: Public CSS test suite mailing list <public-css-testsuite@w3.org>
Message-ID: <224E05E96F1D4EC8B9A1B064433B7474@w3.org>
Thanks. I managed to get the API to work to get at the finer grained data.  

--tobie  


On Thursday, March 28, 2013 at 12:34 AM, "Gérard Talbot" wrote:

> Le Mer 27 mars 2013 15:16, Tobie Langel a écrit :
> > Hi,
> >  
> > I'm trying to get the number of test contained in the following test
> > suites using the Shepherd API:
> >  
> > - CSS3 module: Multi-column layout
>  
> http://test.csswg.org/harness/suite/CSS3-MULTICOL_DEV/
> says
> 124 tests
>  
> > - CSS Color Module Level 3
>  
> http://test.csswg.org/harness/suite/CSS3-COLOR_DEV/
> says
> 54 tests
>  
> > - CSS Transforms
>  
> http://test.csswg.org/harness/suite/CSS3-TRANSFORMS_DEV/
> says
> 928 tests
>  
>  
> > - CSS Backgrounds and Borders Module Level 3
>  
> http://test.csswg.org/harness/suite/CSS3-BACKGROUND_DEV/
> says
> 359 tests
>  
> > - Cascading Style Sheets, level 2 (CSS2) Specification
>  
> http://test.csswg.org/harness/suite/CSS21_DEV/ (nightly-unstable)
> says
> 9735 tests
>  
> ---------------
>  
> Le Mer 27 mars 2013 17:57, Rebecca Hauck a écrit :
> > The landing page of Shepherd summarizes the test counts for each spec:
> >  
> > http://test.csswg.org/shepherd/
>  
> I think such count includes Rejected tests.
>  
> ---------------
>  
> Le Mer 27 mars 2013 18:10, Tobie Langel a écrit :
> > I was curious whether there had been any manual assessment of test
> > coverage.
>  
>  
>  
> Manual assessment of test coverage is either subjective or approximative
> at best. CSS 2.1 test suite most likely has the most complete coverage
> ... and, at the same time, I'm sure there are many tests that we have
> not yet imagined writing. There are even a few spots where the spec
> could be unclear.... therefore untested so far.
>  
> I think we should be creating additional tests on tables.
>  
> Gérard
> --  
> Contributions to the CSS 2.1 test suite:
> http://www.gtalbot.org/BrowserBugsSection/css21testsuite/
>  
> CSS 2.1 Test suite RC6, March 23rd 2011:
> http://test.csswg.org/suites/css2.1/20110323/html4/toc.html
>  
> CSS 2.1 test suite harness:
> http://test.csswg.org/harness/
>  
> Contributing to to CSS 2.1 test suite:
> http://www.gtalbot.org/BrowserBugsSection/css21testsuite/web-authors-contributions-css21-testsuite.html
Received on Wednesday, 27 March 2013 23:37:24 UTC

This archive was generated by hypermail 2.3.1 : Wednesday, 27 March 2013 23:37:26 UTC