W3C home > Mailing lists > Public > www-qa-wg@w3.org > February 2005

Re: Input for Tech Plenary discussion on Multiple Specification Test Suites

From: Patrick Curran <Patrick.Curran@Sun.COM>
Date: Sun, 27 Feb 2005 18:41:18 -0800
To: QAWG <www-qa-wg@w3.org>
Message-id: <422284CE.6000706@sun.com>

Patrick Curran wrote:

> Hi, guys:
> I have volunteered to participate in a panel discussion during the 
> Tech Plenary day next week. Here's the description:
>     Session 4:  Issues in Developing Multiple Specification Test Suites
>     As W3C specifications become more modularized there is a growing
>     number of dependencies between W3C specifications. This panel
>     session will demonstrate and discuss the issues of building test
>     suites that include the requirements of multiple W3C
>     specifications for insuring interoperability.
> All input and suggestions will be welcome. I'll try to circulate a 
> draft of my presentation beforehand, so you have a chance to comment.
> Thanks...
The more I think about this the more I come to the conclusion that 
there's nothing particularly special about multi-spec specs.

Yes: they look like modules, and pose the kinds of problems that modules 
pose (too much variability leads to a lack of interoperability). Yes, if 
you're authoring one you need to think carefully about how you will 
reference external specs (do you really want to commit to support all 
future versions of the referenced spec?)

However, the creation of multi-spec test suites is really about 
developing for re-use. All of the principles that we discuss in the Test 
FAQ apply even more so in multi-spec situations. If you're developing a 
test suite just for one spec then you might be able to cut some corners. 
However, if you're developing for re-use (if you expect your tests to be 
incorporated into another test suite) it's particularly important that 
you clearly state the version of the spec you're testing, that you 
identify the assertions you're testing, that you provide appropriate 
test meta-data, etc.

So: I'm inclined to take this approach for the presentation on 
Wednesday. It will give us an opportunity to preview some of the Test 
FAQ "goood practices", while still addressing the issue at hand.

What do you all think?

Received on Monday, 28 February 2005 02:40:19 UTC

This archive was generated by hypermail 2.3.1 : Tuesday, 6 January 2015 21:14:34 UTC