Re: Fwd: [Markup?] languages for test specification

Thank you Wendy for this good research.

My experience is closest to the last reference Wendy mentioned:  creating
abstract test-requirement patterns as assertions about the good vs. bad
stimulus-response combinations in a suitably elaborated state-trace space. 
"Suitably elaborated" means that before/after transaction distinctions are
made, but some details behind the interface are elided.  Not all details are
exposed, not all details are suppressed.  The view is expanded just enough to
capture all information necessary to evaluate the test-requirement assertions.

For comparable experience in the hardware modeling domain see also

 Forum on Design Languages 2001
 <http://www.ecsi.org/ecsi/fdl/fdl01/tuesday.htm>http://www.ecsi.org/ecsi/f
dl/fdl01/tuesday.htm

Where test point modeling is noted as a specialty-language domain with
currency.

Notes:  

1) the W3C has up until now had an unfortunate bias toward ignoring the
role of
behavior in _defining_ web media, Reference:  Google for "behavior matters
wai-tech-comments".

2) where it comes to specifying and confirming behavior, the requirements
for a
spec-or-test-point-definition language are the same whether the behavior is
realised by a subcircuit or a subprogram.  It's a constraint on the summary
results of an abstracted transaction [encapsulated chunk of computational
activity] either way.

3) note that a lot of the language, in particular the patterns drawn from
mathematics and logic, are shared between the assertions in the specification
and the assertions to be tested in planned test points.  A test plan is
usually
a structured sample of the behavior envelope defined in the specification.
The
specified behavior is covered by a combination of actual test points confirmed
and extrapolation or smoothing based on reasoning about the plausible
continuity or discontinuity of behavior.  The construction rules for
specification assertions and test requirements are much the same, but
generally
some particulars in the test points are specializations of the particulars in
the specification assertions.

The specification is composed of a world model and a collection of assertions
relative to that world model.  A test plan is composed of a refined world
model
[adding test equipment as the specific environment in which tests are
performed
-- software test harnesses are 'equipment' in this sense] and a collection of
refined assertions to be confirmed empirically in that context.

4) I put 'markup' in question because XML is usable to create fully formal
languages as well as markup languages like [X]HTML where most of the
information is informally encoded with a light wrapping of more formal
markings.  Of course test points calling for human assessment [does this ALT
text fit?] have entrained natural language definitions of the assertion to be
checked.

Al

At 04:59 AM 2001-10-10 , Daniel Dardailler wrote:
>
>This was sent to the WAI Evaluation and Repair Tools group by Wendy, I 
>think it's useful info for QA record as well.
>
>> I took an action at the F2F to chase up markup languages for test 
>> specification. I had vaguely recalled hearing something that NIST was 
>> working on. Today in my searches, I quickly stumbled on the DOM test suite 
>> markup language that Dimitris and others spoke of at the ERT/PF F2F at the 
>> tech plenary last February.
>> 
>> Here is a bit about it in the FAQ.
>>
<http://www.w3.org/DOM/Test/Documents/DOMTSFAQ.html#usingXML>http://www.w3.o
rg/DOM/Test/Documents/DOMTSFAQ.html#usingXML
>> 
>> Here's a message from Dimitris from May
>>
<http://lists.w3.org/Archives/Public/www-dom-ts/2001May/0067.html>http://lis
ts.w3.org/Archives/Public/www-dom-ts/2001May/0067.html
>> that includes a DTD
>>
<http://lists.w3.org/Archives/Public/www-dom-ts/2001May/att-0067/01-methods
.dtd>http://lists.w3.org/Archives/Public/www-dom-ts/2001May/att-0067/01-meth
ods.dtd
>> 
>> He attended an ERT call on 28 May 2001
>>
<http://lists.w3.org/Archives/Public/w3c-wai-er-ig/2001May/0094.html>http://
lists.w3.org/Archives/Public/w3c-wai-er-ig/2001May/0094.html
>> 
>> I didn't catch it before, but now I realize how the DOM TS ML and EARL 
>> compliment each other.  Charles saw it when he said, "CMN Difference is 
>> instead of generating "pass or fail" you are generating EARL." during the 
>> 28 May telecon.
>> 
>> 
>> A wider search brought up the following abstracts.
>> TTCN-3 - A new Test Specification Language for Black-Box Testing of 
>> Distributed Systems.
>>
<http://www.itm.mu-luebeck.de/english/publications/Abstract_ttcn-3.html>http
://www.itm.mu-luebeck.de/english/publications/Abstract_ttcn-3.html
>> Towards the new Test Specification and Implementation Language 'TelCom TSL'
>>
<http://iamwww.unibe.ch/~rvswww/Publikationen/Grabowski/GI-ITG-1995/abstract
>http://iamwww.unibe.ch/~rvswww/Publikationen/Grabowski/GI-ITG-1995/abstract. 
>> html
>> And a paper: 
>>
<http://citeseer.nj.nec.com/cache/papers2/cs/16526/http:zSzzSzwww.itm.mu-lu
eb>http://citeseer.nj.nec.com/cache/papers2/cs/16526/http:zSzzSzwww.itm.mu-l
ueb 
>> eck.dezSzpublicationszSzttcn3zSzGrabowski.pdf/grabowski00ttcn.pdf
>> 
>> Test Specification DTD
>> This DTD defines the format for test specifiers which define component 
>> tests and can be executed by the Test Pattern Verifier.  It describes the 
>> tests as well as the results.  The end-to-end process is to verify 
>> components so that they may be certified and then made available through a 
>> component server.
>>
<http://ciips.ee.uwa.edu.au/Research/SCL/Docs.html>http://ciips.ee.uwa.edu.a
u/Research/SCL/Docs.html
>> Part of the SCL Component Test Bed Specification 
>> <http://xml.coverpages.org/scl.html>http://xml.coverpages.org/scl.html
>> 
>> Interesting tangent: IMS has a language to report results of student 
>> assessments.  Another possible use case for EARL, although they are
already 
>> using their own XML dialect.
>> <http://www.imsproject.org/question/>http://www.imsproject.org/question/
>> 
>> "ADL, or the Assertion Definition Language, is a formal grammar for 
>> describing the behavior of interfaces. This very general concept can be 
>> applied to any interface for which the behavior can be described. The 
>> purpose of this grammar is two-fold. First, it permits the translation of 
>> the formal grammar into natural languages such as English and Japanese. 
>> Second, it permits the automatic translation of the formal grammar into 
>> tests that will evaluate the behavior of an implementation of the
interface 
>> being described. "
>> <http://adl.opengroup.org/>http://adl.opengroup.org/
>> 
>> Found a variety of sites, books, etc. devoted to Web testing methods and 
>> definitions.  Investigating these further.
>> 
>> --wendy
>> --
>> wendy a chisholm
>> world wide web consortium
>> web accessibility initiative
>> seattle, wa usa
>> /--
>

Received on Thursday, 11 October 2001 12:36:41 UTC