Re: Test suite for services approach

Felix,

Thanks for your proposal for a test suite design.

I guess what you are providing is the implementation report (based on 
the Test suite).

- The test suite is a collection a testcases.
- The implementation is a table that shows which testcases of the test 
suite are passed (or failed) by each implementation.

The distinction is important because
- the purpose of the implementation reports is to show to the Director 
that the WG has fulfill the CR exit criteria, and therefore move to PR.
- The test suite may allow further implementations (for example after PR 
or REC, to test their product.

Here are a few examples I have experienced in different WGs

* the SYMM WG

- testsuite
http://www.w3.org/2005/SMIL21/testsuite/New-SMIL21/NewS21functionalities.html#Layout
- impementation report
http://www.w3.org/2005/SMIL21/SMIL21-implementation-result.html

* the webCGM WG

- testsuite
http://www.w3.org/Graphics/WebCGM/2009/WebCGM21/testsuite21.html
- impementation report
http://www.w3.org/Graphics/WebCGM/2009/WebCGM21/new21-matrix.html












  Sasaki wrote:
> Hi all,
> 
> at http://fabday.fh-potsdam.de/~sasaki/mawg/mawg-testsuite-proposal.html 
> there is a proposal for a test suite design, see ACTION-213 
> <http://www.w3.org/2008/WebVideo/Annotations/track/actions/213>. It 
> contains only a pattern for a test for one property, but the pattern 
> should be applicable to all properties:
> - give each property a row in the table
> - have a column with the source. If we have several formats to be 
> tested, we would need several rows for the property
> - have a column with the expected results.
> - have further columns for other results.
> Please note the proposed directory structure which you can see by 
> hovering other the links, with directories for input files, expected 
> output, and actual output for each implementation.
> 
> Best,
> 
> Felix

Received on Friday, 19 February 2010 09:56:25 UTC