Test Cases, TPDL and Formalisms in EARL

The reference mentioned for the DOM TS DTD is substantially out of date.  The DTD and Schema for a specific DOM specification are generated, so there aren't copies in the CVS.  For discussion I've generated a DTD and schema for the DOM Level 1 at http://home.houston.rr.com/curta/domtest/dom1.dtd and http://home.houston.rr.com/curta/domtest/dom1.xsd

The metadata in the DOM TS is patterned after Dublin Core expressed in XML, however the group decided to keep all the elements in one namespace for authoring simplicity.

The DOM TS language is designed to be concise as possible without adding excessive complexity to the processing code.

Here is an example test case

<?xml version="1.0" encoding="UTF-8"?>
<!--
Copyright (c) 2001 World Wide Web Consortium,
(Massachusetts Institute of Technology, Institut National de
Recherche en Informatique et en Automatique, Keio University). All
Rights Reserved. This program is distributed under the W3C's Software
Intellectual Property License. This program is distributed in the
hope that it will be useful, but WITHOUT ANY WARRANTY; without even
the implied warranty of MERCHANTABILITY or FITNESS FOR A PARTICULAR
PURPOSE.
See W3C License http://www.w3.org/Consortium/Legal/ for more details.
--><!DOCTYPE test SYSTEM "dom1.dtd">

<test xmlns="http://www.w3.org/2001/DOM-Test-Suite/Level-1" name="attrname">
<metadata>
<title>attrName</title>
<creator>NIST</creator>
<description>
The getNodeName() method of an Attribute node. 
Retrieve the attribute named street from the last 
child of of the second employee and examine its 
NodeName. This test uses the getNamedItem(name) method from the NamedNodeMap 
interface.
</description>
<contributor>Mary Brady</contributor>
<date qualifier="created">2001-08-17</date>
<!-- Node.nodeName -->
<subject resource="http://www.w3.org/TR/1998/REC-DOM-Level-1-19981001/level-one-core#ID-F68D095"/>
<!-- Attr.name -->
<subject resource="http://www.w3.org/TR/1998/REC-DOM-Level-1-19981001/level-one-core#ID-1112119403"/>
</metadata>
<var name="doc" type="Document"/>
<var name="addressList" type="NodeList"/>
<var name="testNode" type="Node"/>
<var name="attributes" type="NamedNodeMap"/>
<var name="streetAttr" type="Attr"/>
<var name="name" type="DOMString"/>
<load var="doc" href="staff.xml" willBeModified="false"/>
<getElementsByTagName interface="Document" obj="doc" var="addressList" tagname="&quot;address&quot;"/>
<item interface="NodeList" obj="addressList" var="testNode" index="1"/>
<attributes obj="testNode" var="attributes"/>
<getNamedItem obj="attributes" var="streetAttr" name="&quot;street&quot;"/>
<nodeName obj="streetAttr" var="name"/>
<assertEquals actual="name" expected="&quot;street&quot;" id="nodeName" ignoreCase="false"/>
<name obj="streetAttr" var="name" interface="Attr"/>
<assertEquals actual="name" expected="&quot;street&quot;" id="name" ignoreCase="false"/>
</test>


The metadata section identifies the title, creator, and contributors to the tests.  

The subjects are references to particular passages in the spec that the test exercises and are used in establishing the test matrix and might be the same concept as "Test Points".  All interfaces, attributes and properties had an ID attribute in the specificiation.  XPointer expressions like:

<subject resource="http://www.w3.org/TR/1998/REC-DOM-Level-1-19981001/level-one-core#xpointer(id('ID-E5CBA7FB')/raises/exception[@name='DOMException']/descr/p[substring-before(.,':')='INDEX_SIZE_ERR'])"/>


are used to identity passages in the prose that describe particular exceptions that should be raised.

A list of "subjects" in a DOM specification is extracted in RDF/XML by a style-sheet, dom-subjects.xml.  For discussion, an example of a subject list for DOM 1 spec is http://home.houston.rr.com/curta/domtest/dom1-subjects.xml.gz  

The stylesheet considers any interface, method, attribute or mention of a specific exception code as a potential test subject.  As mentioned previously, exception codes raised by a specific method simply appear in the prose description.  There are also a lot of other passages in the prose that are conceptually test points but cannot be identified structually.  Anything that could encourage these "implicit" test points to be explicitly identified would be helpful.  From the DOM specifications authoring side, I would expect they would just add either an element to their DTD that could identify currently implicit test points.  TPDL could be the output of a next generation dom-subjects.xml.

Each assertion in the test is required to have an ID attribute, so that a failure of a particular assertion could be identified.

Currently, the tests are run by xUnit test harnesses (JUnit, JSUnit for now, NUnit, PyUnit, etc forthcoming) so test reporting is under their control.

I have requested some guidance on the use of EARL in RDF/XML syntax, but haven't gotten much help.  I'm sure N3 is convienient once you get over the learning curve, but the lack of RDF/XML examples in the EARL documentation makes it very difficult to pick up when you are only an XML guru and not an RDF guru.  If EARL was reasonable to implement and added value, then I would try to get it incorporated as a reporting option in the xUnit testing frameworks used by the DOM tests.