W3C home > Mailing lists > Public > www-qa-wg@w3.org > June 2003

certification footnote [was: Re: The bad side of test cases]

From: Lofton Henderson <lofton@rockynet.com>
Date: Mon, 23 Jun 2003 14:06:53 -0600
Message-Id: <5.1.0.14.2.20030623135005.02e0bde0@rockynet.com>
To: www-qa-wg@w3.org

[...switched list to WG...]

Karl's reference reminds me of one other comment I wanted to add to our 
Crete discussion on certification.  I said, and the minutes recorded, a 
couple of things about the the suitability of SVG (maybe Tiny/cell phone) 
as a candidate for a pilot project.  (Assuming that there were to be such a 
project, which has NOT been decided.)

There is a third problem which affects even SVG Tiny...

At 05:41 AM 6/22/03 -0400, Karl Dubost wrote:

>In Surfin safari, Dave Hyatt, developer of Safari and member of the CSS WG 
>has said:
>http://weblogs.mozillazine.org/hyatt/archives/2003_06.html#003539
>
>[...]
>Read the full article. In the completion of a Test Suite with test cases, 
>how do we define the depth of it, where does it stop? I pointed to this 
>comment because we have started to write the Test Guidelines.

In the article (or was it his preceding article?) Dave also says:

"What irritates me about these charts is how inaccurate they frequently end 
up being. They are inherently biased towards feature breadth and not 
feature depth, and in the real world feature depth is so much more important. "

This is true for SVG Tiny.  The SVG test suite is "Basic Effectivity", 
meaning breadth-first (and not much depth).  BE is useful (as the author 
says) for diagnosing which functional areas have been implemented and which 
have not.  But they do not help much to diagnose how thorough (and correct) 
is the implementation of each feature.

Before such test suites could be candidates for certification use, they 
would have to be expanded depth-wise.  For something like SVG, we estimated 
that a DT suite ("detailed") would be at least 10 times as many tests as 
the BE suite.

-Lofton.
Received on Monday, 23 June 2003 16:16:05 GMT

This archive was generated by hypermail 2.2.0 + w3c-0.30 : Thursday, 9 June 2005 12:13:14 GMT