- From: Al Gilman <asgilman@iamdigex.net>
- Date: Thu, 11 Oct 2001 11:45:45 -0400
- To: www-qa@w3.org
- Cc: danield@w3.org, Wendy A Chisholm <wendy@w3.org>
Thank you Wendy for this good research. My experience is closest to the last reference Wendy mentioned: creating abstract test-requirement patterns as assertions about the good vs. bad stimulus-response combinations in a suitably elaborated state-trace space. "Suitably elaborated" means that before/after transaction distinctions are made, but some details behind the interface are elided. Not all details are exposed, not all details are suppressed. The view is expanded just enough to capture all information necessary to evaluate the test-requirement assertions. For comparable experience in the hardware modeling domain see also Forum on Design Languages 2001 <http://www.ecsi.org/ecsi/fdl/fdl01/tuesday.htm>http://www.ecsi.org/ecsi/f dl/fdl01/tuesday.htm Where test point modeling is noted as a specialty-language domain with currency. Notes: 1) the W3C has up until now had an unfortunate bias toward ignoring the role of behavior in _defining_ web media, Reference: Google for "behavior matters wai-tech-comments". 2) where it comes to specifying and confirming behavior, the requirements for a spec-or-test-point-definition language are the same whether the behavior is realised by a subcircuit or a subprogram. It's a constraint on the summary results of an abstracted transaction [encapsulated chunk of computational activity] either way. 3) note that a lot of the language, in particular the patterns drawn from mathematics and logic, are shared between the assertions in the specification and the assertions to be tested in planned test points. A test plan is usually a structured sample of the behavior envelope defined in the specification. The specified behavior is covered by a combination of actual test points confirmed and extrapolation or smoothing based on reasoning about the plausible continuity or discontinuity of behavior. The construction rules for specification assertions and test requirements are much the same, but generally some particulars in the test points are specializations of the particulars in the specification assertions. The specification is composed of a world model and a collection of assertions relative to that world model. A test plan is composed of a refined world model [adding test equipment as the specific environment in which tests are performed -- software test harnesses are 'equipment' in this sense] and a collection of refined assertions to be confirmed empirically in that context. 4) I put 'markup' in question because XML is usable to create fully formal languages as well as markup languages like [X]HTML where most of the information is informally encoded with a light wrapping of more formal markings. Of course test points calling for human assessment [does this ALT text fit?] have entrained natural language definitions of the assertion to be checked. Al At 04:59 AM 2001-10-10 , Daniel Dardailler wrote: > >This was sent to the WAI Evaluation and Repair Tools group by Wendy, I >think it's useful info for QA record as well. > >> I took an action at the F2F to chase up markup languages for test >> specification. I had vaguely recalled hearing something that NIST was >> working on. Today in my searches, I quickly stumbled on the DOM test suite >> markup language that Dimitris and others spoke of at the ERT/PF F2F at the >> tech plenary last February. >> >> Here is a bit about it in the FAQ. >> <http://www.w3.org/DOM/Test/Documents/DOMTSFAQ.html#usingXML>http://www.w3.o rg/DOM/Test/Documents/DOMTSFAQ.html#usingXML >> >> Here's a message from Dimitris from May >> <http://lists.w3.org/Archives/Public/www-dom-ts/2001May/0067.html>http://lis ts.w3.org/Archives/Public/www-dom-ts/2001May/0067.html >> that includes a DTD >> <http://lists.w3.org/Archives/Public/www-dom-ts/2001May/att-0067/01-methods .dtd>http://lists.w3.org/Archives/Public/www-dom-ts/2001May/att-0067/01-meth ods.dtd >> >> He attended an ERT call on 28 May 2001 >> <http://lists.w3.org/Archives/Public/w3c-wai-er-ig/2001May/0094.html>http:// lists.w3.org/Archives/Public/w3c-wai-er-ig/2001May/0094.html >> >> I didn't catch it before, but now I realize how the DOM TS ML and EARL >> compliment each other. Charles saw it when he said, "CMN Difference is >> instead of generating "pass or fail" you are generating EARL." during the >> 28 May telecon. >> >> >> A wider search brought up the following abstracts. >> TTCN-3 - A new Test Specification Language for Black-Box Testing of >> Distributed Systems. >> <http://www.itm.mu-luebeck.de/english/publications/Abstract_ttcn-3.html>http ://www.itm.mu-luebeck.de/english/publications/Abstract_ttcn-3.html >> Towards the new Test Specification and Implementation Language 'TelCom TSL' >> <http://iamwww.unibe.ch/~rvswww/Publikationen/Grabowski/GI-ITG-1995/abstract >http://iamwww.unibe.ch/~rvswww/Publikationen/Grabowski/GI-ITG-1995/abstract. >> html >> And a paper: >> <http://citeseer.nj.nec.com/cache/papers2/cs/16526/http:zSzzSzwww.itm.mu-lu eb>http://citeseer.nj.nec.com/cache/papers2/cs/16526/http:zSzzSzwww.itm.mu-l ueb >> eck.dezSzpublicationszSzttcn3zSzGrabowski.pdf/grabowski00ttcn.pdf >> >> Test Specification DTD >> This DTD defines the format for test specifiers which define component >> tests and can be executed by the Test Pattern Verifier. It describes the >> tests as well as the results. The end-to-end process is to verify >> components so that they may be certified and then made available through a >> component server. >> <http://ciips.ee.uwa.edu.au/Research/SCL/Docs.html>http://ciips.ee.uwa.edu.a u/Research/SCL/Docs.html >> Part of the SCL Component Test Bed Specification >> <http://xml.coverpages.org/scl.html>http://xml.coverpages.org/scl.html >> >> Interesting tangent: IMS has a language to report results of student >> assessments. Another possible use case for EARL, although they are already >> using their own XML dialect. >> <http://www.imsproject.org/question/>http://www.imsproject.org/question/ >> >> "ADL, or the Assertion Definition Language, is a formal grammar for >> describing the behavior of interfaces. This very general concept can be >> applied to any interface for which the behavior can be described. The >> purpose of this grammar is two-fold. First, it permits the translation of >> the formal grammar into natural languages such as English and Japanese. >> Second, it permits the automatic translation of the formal grammar into >> tests that will evaluate the behavior of an implementation of the interface >> being described. " >> <http://adl.opengroup.org/>http://adl.opengroup.org/ >> >> Found a variety of sites, books, etc. devoted to Web testing methods and >> definitions. Investigating these further. >> >> --wendy >> -- >> wendy a chisholm >> world wide web consortium >> web accessibility initiative >> seattle, wa usa >> /-- >
Received on Thursday, 11 October 2001 12:36:41 UTC