W3C home > Mailing lists > Public > www-qa-wg@w3.org > December 2003

Re: Thursday's Discussion

From: Mark Skall <mark.skall@nist.gov>
Date: Tue, 16 Dec 2003 15:56:28 -0500
Message-Id: <>
To: Lofton Henderson <lofton@rockynet.com>
Cc: www-qa-wg@w3.org

>This is a side issue, but related -- if we are going to look at evaluation 
>criteria such as "specific enough to generate tests", it helps to have a 
>concept of what tests for SpecGL might look like.
>What do you think of when you think of "tests"?
>As we discussed earlier (but I'm not sure of our level of consensus), a 
>manual test could be as simple as a per-conformance-requirement (or 
>per-TA) question, "Does the spec ...blah...?  Choice of:  NO; YES at 
>[...fill-in-link-to-spec-loc...]"  "Blah" is effectively the statement of 
>the conformance requirement.
>Are you thinking of manual testing like this?  Or are you thinking of some 
>sort of automated SpecGL tester?  Or ...?

I am thinking of a precise way to check to see if the requirement is 
met.  In the case of SpecGL, in my mind, this would be a completely manual 
process.  The key is precision, not automation.  If the requirements (and 
TAs) are not precise, you will end up with different interpretations and 
different tests.

>Can you also supply a list of the ones that you found to be problematic 
>(and why)?

Without going over the reasons in detail (we will do this at the telcon), 
the following are checkpoints where I needed more info or made assumptions 
in order to produce the assertions:

1.1, 1.2, 2.3, 4.1, 7.2, 8.2, 8.5, 9.2, 10.1, 10.2

Some of these are fairly trivial and some are a little more 
substantive.  However, as we all know, when it comes to conformance,  even 
the most trivial requirements must be precisely specified.

Mark Skall
Chief, Software Diagnostics and Conformance Testing Division
Information Technology Laboratory
National Institute of Standards and Technology (NIST)
100 Bureau Drive, Stop 8970
Gaithersburg, MD 20899-8970

Voice: 301-975-3262
Fax:   301-590-9174
Email: skall@nist.gov
Received on Tuesday, 16 December 2003 16:01:51 UTC

This archive was generated by hypermail 2.3.1 : Tuesday, 6 January 2015 21:14:31 UTC