Test Cases and Comparisons Re: Minutes for Teleconference on 30 August, 2006

Hi Shadi and ERT WG,

Le 30 août 06 à 23:56, Shadi Abou-Zahra a écrit :
> Please find the minutes for the teleconference on 30 August, 2006:
> <http://www.w3.org/2006/08/30-er-minutes>

[[[
SAZ: Main issue is missing TCDL which limits the interopability to  
say things are the same - we can get by without that and even in the  
worst case you can still work with the general superclass of  
testablestatement ... We hope people would be more consistent and a  
TCDL will arrive in the future
]]] -- http://www.w3.org/2006/08/30-er-minutes
Wed, 30 Aug 2006 14:51:38 GMT

In W3C [QA dev][1], we are developing a [Test Suite driver for CSS  
Validator][2]. The intent is to test the validator itself.

The way it is working right now, there are two files
     - a Test Case (CSS file or html file valid or not valid)
     - a predefined output from the validator
     - a description of the test

1. The test case is sent to the validator for validation
2. The validator returns a message.
3. The message is compared to the predefined output.
4. From the comparison a pass or fail is issued

We were discussing with Olivier two days ago on how we would express  
that with EARL, aka the link to the file, the supposed result, the  
description and the Pass, Fail results.

I remember also that in SVG, they were using comparison images for  
their Test Suite.
     1. An image reference is given in PNG
     2. a file in SVG is sent to a renderer.
     3. A bitmap of this rendering is generated,
     4. the bitmap and the reference image are compared.


I see the last Editor's Draft is 18th of July[3].
Could you give an example on how we would express this kind of  
pattern in EARL, if possible?



[1]: http://www.w3.org/QA/Tools/qa-dev/
[2]: http://dev.w3.org/cvsweb/2002/css-validator/autotest/
[3]: http://www.w3.org/WAI/ER/EARL10/WD-EARL10-Schema-20060718


-- 
Karl Dubost - http://www.w3.org/People/karl/
W3C Conformance Manager, QA Activity Lead
   QA Weblog - http://www.w3.org/QA/
      *** Be Strict To Be Cool ***

Received on Thursday, 31 August 2006 01:29:08 UTC