W3C home > Mailing lists > Public > www-qa@w3.org > October 2001

Re: Fwd: [Markup?] languages for test specification

From: Al Gilman <asgilman@iamdigex.net>
Date: Thu, 11 Oct 2001 11:45:45 -0400
Message-Id: <200110111537.LAA3288268@smtp1.mail.iamworld.net>
To: www-qa@w3.org
Cc: danield@w3.org, Wendy A Chisholm <wendy@w3.org>
Thank you Wendy for this good research.

My experience is closest to the last reference Wendy mentioned:  creating
abstract test-requirement patterns as assertions about the good vs. bad
stimulus-response combinations in a suitably elaborated state-trace space. 
"Suitably elaborated" means that before/after transaction distinctions are
made, but some details behind the interface are elided.  Not all details are
exposed, not all details are suppressed.  The view is expanded just enough to
capture all information necessary to evaluate the test-requirement assertions.

For comparable experience in the hardware modeling domain see also

 Forum on Design Languages 2001

Where test point modeling is noted as a specialty-language domain with


1) the W3C has up until now had an unfortunate bias toward ignoring the
role of
behavior in _defining_ web media, Reference:  Google for "behavior matters

2) where it comes to specifying and confirming behavior, the requirements
for a
spec-or-test-point-definition language are the same whether the behavior is
realised by a subcircuit or a subprogram.  It's a constraint on the summary
results of an abstracted transaction [encapsulated chunk of computational
activity] either way.

3) note that a lot of the language, in particular the patterns drawn from
mathematics and logic, are shared between the assertions in the specification
and the assertions to be tested in planned test points.  A test plan is
a structured sample of the behavior envelope defined in the specification.
specified behavior is covered by a combination of actual test points confirmed
and extrapolation or smoothing based on reasoning about the plausible
continuity or discontinuity of behavior.  The construction rules for
specification assertions and test requirements are much the same, but
some particulars in the test points are specializations of the particulars in
the specification assertions.

The specification is composed of a world model and a collection of assertions
relative to that world model.  A test plan is composed of a refined world
[adding test equipment as the specific environment in which tests are
-- software test harnesses are 'equipment' in this sense] and a collection of
refined assertions to be confirmed empirically in that context.

4) I put 'markup' in question because XML is usable to create fully formal
languages as well as markup languages like [X]HTML where most of the
information is informally encoded with a light wrapping of more formal
markings.  Of course test points calling for human assessment [does this ALT
text fit?] have entrained natural language definitions of the assertion to be


At 04:59 AM 2001-10-10 , Daniel Dardailler wrote:
>This was sent to the WAI Evaluation and Repair Tools group by Wendy, I 
>think it's useful info for QA record as well.
>> I took an action at the F2F to chase up markup languages for test 
>> specification. I had vaguely recalled hearing something that NIST was 
>> working on. Today in my searches, I quickly stumbled on the DOM test suite 
>> markup language that Dimitris and others spoke of at the ERT/PF F2F at the 
>> tech plenary last February.
>> Here is a bit about it in the FAQ.
>> Here's a message from Dimitris from May
>> that includes a DTD
>> He attended an ERT call on 28 May 2001
>> I didn't catch it before, but now I realize how the DOM TS ML and EARL 
>> compliment each other.  Charles saw it when he said, "CMN Difference is 
>> instead of generating "pass or fail" you are generating EARL." during the 
>> 28 May telecon.
>> A wider search brought up the following abstracts.
>> TTCN-3 - A new Test Specification Language for Black-Box Testing of 
>> Distributed Systems.
>> Towards the new Test Specification and Implementation Language 'TelCom TSL'
>> html
>> And a paper: 
>> eck.dezSzpublicationszSzttcn3zSzGrabowski.pdf/grabowski00ttcn.pdf
>> Test Specification DTD
>> This DTD defines the format for test specifiers which define component 
>> tests and can be executed by the Test Pattern Verifier.  It describes the 
>> tests as well as the results.  The end-to-end process is to verify 
>> components so that they may be certified and then made available through a 
>> component server.
>> Part of the SCL Component Test Bed Specification 
>> <http://xml.coverpages.org/scl.html>http://xml.coverpages.org/scl.html
>> Interesting tangent: IMS has a language to report results of student 
>> assessments.  Another possible use case for EARL, although they are
>> using their own XML dialect.
>> <http://www.imsproject.org/question/>http://www.imsproject.org/question/
>> "ADL, or the Assertion Definition Language, is a formal grammar for 
>> describing the behavior of interfaces. This very general concept can be 
>> applied to any interface for which the behavior can be described. The 
>> purpose of this grammar is two-fold. First, it permits the translation of 
>> the formal grammar into natural languages such as English and Japanese. 
>> Second, it permits the automatic translation of the formal grammar into 
>> tests that will evaluate the behavior of an implementation of the
>> being described. "
>> <http://adl.opengroup.org/>http://adl.opengroup.org/
>> Found a variety of sites, books, etc. devoted to Web testing methods and 
>> definitions.  Investigating these further.
>> --wendy
>> --
>> wendy a chisholm
>> world wide web consortium
>> web accessibility initiative
>> seattle, wa usa
>> /--
Received on Thursday, 11 October 2001 12:36:41 UTC

This archive was generated by hypermail 2.3.1 : Tuesday, 6 January 2015 20:40:28 UTC