Fw: Test Cases, TPDL and Formalisms in EARL

----- Original Message -----
From: Curt Arnold
To: w3c-qa@w3.org
Cc: www-dom-ts@w3.org
Sent: Tuesday, October 16, 2001 2:04 AM
Subject: Test Cases, TPDL and Formalisms in EARL


The reference mentioned for the DOM TS DTD is substantially out of date.
The DTD and Schema for a specific DOM specification are generated, so there
aren't copies in the CVS.  For discussion I've generated a DTD and schema
for the DOM Level 1 at http://home.houston.rr.com/curta/domtest/dom1.dtd and
http://home.houston.rr.com/curta/domtest/dom1.xsd

The metadata in the DOM TS is patterned after Dublin Core expressed in XML,
however the group decided to keep all the elements in one namespace for
authoring simplicity.

The DOM TS language is designed to be concise as possible without adding
excessive complexity to the processing code.

Here is an example test case

<?xml version="1.0" encoding="UTF-8"?>
<!--
Copyright (c) 2001 World Wide Web Consortium,
(Massachusetts Institute of Technology, Institut National de
Recherche en Informatique et en Automatique, Keio University). All
Rights Reserved. This program is distributed under the W3C's Software
Intellectual Property License. This program is distributed in the
hope that it will be useful, but WITHOUT ANY WARRANTY; without even
the implied warranty of MERCHANTABILITY or FITNESS FOR A PARTICULAR
PURPOSE.
See W3C License http://www.w3.org/Consortium/Legal/ for more details.
--><!DOCTYPE test SYSTEM "dom1.dtd">

<test xmlns="http://www.w3.org/2001/DOM-Test-Suite/Level-1" name="attrname">
<metadata>
<title>attrName</title>
<creator>NIST</creator>
<description>
The getNodeName() method of an Attribute node.
Retrieve the attribute named street from the last
child of of the second employee and examine its
NodeName. This test uses the getNamedItem(name) method from the NamedNodeMap
interface.
</description>
<contributor>Mary Brady</contributor>
<date qualifier="created">2001-08-17</date>
<!-- Node.nodeName -->
<subject
resource="http://www.w3.org/TR/1998/REC-DOM-Level-1-19981001/level-one-core#
ID-F68D095"/>
<!-- Attr.name -->
<subject
resource="http://www.w3.org/TR/1998/REC-DOM-Level-1-19981001/level-one-core#
ID-1112119403"/>
</metadata>
<var name="doc" type="Document"/>
<var name="addressList" type="NodeList"/>
<var name="testNode" type="Node"/>
<var name="attributes" type="NamedNodeMap"/>
<var name="streetAttr" type="Attr"/>
<var name="name" type="DOMString"/>
<load var="doc" href="staff.xml" willBeModified="false"/>
<getElementsByTagName interface="Document" obj="doc" var="addressList"
tagname="&quot;address&quot;"/>
<item interface="NodeList" obj="addressList" var="testNode" index="1"/>
<attributes obj="testNode" var="attributes"/>
<getNamedItem obj="attributes" var="streetAttr" name="&quot;street&quot;"/>
<nodeName obj="streetAttr" var="name"/>
<assertEquals actual="name" expected="&quot;street&quot;" id="nodeName"
ignoreCase="false"/>
<name obj="streetAttr" var="name" interface="Attr"/>
<assertEquals actual="name" expected="&quot;street&quot;" id="name"
ignoreCase="false"/>
</test>

The metadata section identifies the title, creator, and contributors to the
tests.
The subjects are references to particular passages in the spec that the test
exercises and are used in establishing the test matrix and might be the same
concept as "Test Points".  All interfaces, attributes and properties had an
ID attribute in the specificiation.  XPointer expressions like:
<subject
resource="http://www.w3.org/TR/1998/REC-DOM-Level-1-19981001/level-one-core#
xpointer(id('ID-E5CBA7FB')/raises/exception[@name='DOMException']/descr/p[su
bstring-before(.,':')='INDEX_SIZE_ERR'])"/>

are used to identity passages in the prose that describe particular
exceptions that should be raised.
A list of "subjects" in a DOM specification is extracted in RDF/XML by a
style-sheet, dom-subjects.xml.  For discussion, an example of a subject list
for DOM 1 spec is
http://home.houston.rr.com/curta/domtest/dom1-subjects.xml.gz
The stylesheet considers any interface, method, attribute or mention of a
specific exception code as a potential test subject.  As mentioned
previously, exception codes raised by a specific method simply appear in the
prose description.  There are also a lot of other passages in the prose that
are conceptually test points but cannot be identified structually.  Anything
that could encourage these "implicit" test points to be explicitly
identified would be helpful.  From the DOM specifications authoring side, I
would expect they would just add either an element to their DTD that could
identify currently implicit test points.  TPDL could be the output of a next
generation dom-subjects.xml.
Each assertion in the test is required to have an ID attribute, so that a
failure of a particular assertion could be identified.
Currently, the tests are run by xUnit test harnesses (JUnit, JSUnit for now,
NUnit, PyUnit, etc forthcoming) so test reporting is under their control.
I have requested some guidance on the use of EARL in RDF/XML syntax, but
haven't gotten much help.  I'm sure N3 is convienient once you get over the
learning curve, but the lack of RDF/XML examples in the EARL documentation
makes it very difficult to pick up when you are only an XML guru and not an
RDF guru.  If EARL was reasonable to implement and added value, then I would
try to get it incorporated as a reporting option in the xUnit testing
frameworks used by the DOM tests.

Received on Tuesday, 16 October 2001 03:07:05 UTC