RE: [General] Language-independent test representation

> -----Original Message-----
> From: Mary Brady [mailto:mbrady@nist.gov]
> Sent: Monday, April 23, 2001 10:59 AM
> To: Arnold, Curt; www-dom-ts@w3.org
> Subject: Re: [General] Language-independent test representation
> 
> 
> We have been working on the XML representation of the test
> scenarios and an XSLT transformation -- We currently have
> this working for all of our Java-based tests and have a java
> translator.  I've included an example called node.xml, the
> transformation, called generate.xsl, the dtd, methods.dtd, and
> a translated file, node.java.  Right now, we are working on
> generating javadoc style comments as well, and cleaning up
> parantheses that we don't need, etc.  We will also have an
> ECMAScript translator -- probably by the end of the week.
> 
> Joe is right in that we have ended up having to support much of
> the functionality of a programming language, including type
> definitions, method calls, for/while loops, assignments, etc, but
> we have been able to encapsulate all of our current tests and
> don't expect to have any problem generating ECMAScript code.
> This approach also has the advantage of being able to generate
> code for other languages, such as Perl, Python, C, C++, simply
> by adding one more style sheet and a set of language def's for the
> dom calls.  It also affords us the ability to maintain one common
> code base.

Broken record follows....

I think that I've mentioned that I have been able to get CppUnit
to run for Xerces-C, again like the JavaScript port, done by fairly
simple changes that could be automated by AWK or something similar.  
It would be interesting to see how difficult it would be to port the tests to
the Perl or Python variants of xUnit, but I'm not fluent with those
languages.  If it does require something beyond regular expression 
then there are plenty of options to parse the Java source and generate
equivalent code without inventing a new language.

The sample node.xml still seems to maintain a limitation of one string
assertion per test.  If we are bringing in tests from other sources,
this limitation could be barrier to integration.  Plus, it doesn't allow
any tests of object identity or other type specific tests or 
distinguishing a method returning a string containing "null" vs a null string.

You would never expect people to formulate tests in this language.  You
would always expect that the development of test would always be done
in some other language and then converted.    Since there are plenty 
of ways to convert Java based tests to a possible language independent 
format, trying to fix the format at this time seems premature and a
distraction.  I think the priority should be to integrate tests from 
other sources into the JUnit framework and then continue to refine 
the generation of the equivalent JSUnit, CppUnit, et al tests.

> We do have a couple of questions at this point and are looking for
> input.  First, our tests load an xml file and associated dtd under a
> couple of scenarios -- so, we need support for validating, 
> non-validating,
> entities, expanded and not expanded, whitespace, retained, stripped.

I've hacked the tests to support a couple of different scenarios
(entity expansion, ignoring element content whitespace and ignoring DTD's).  
Basically, I added a series of property accessors to the base test class 
and check these within the tests, typically either changing the node 
offsets before the tests or changing the assertions after the tests.


> Any suggestions on whether this should be handled inside a test, or
> more generically by an associated test harness?  Inside the test would
> give us stand-alone tests but would force the developer to change the
> load mechanism for every test...Outside forces the use of a driver.
> We have a similar problem with comparing expected results and
> actual results -- should it be done inside the test or with 
> an associated
> harness.

I don't think the logic to adjust to different DOM properties
can be accomplished outside of the tests.  For example, you couldn't 
safely just divide all the node indexes by 2 if you were ignoring whitespace, 
though it would work for most of the NIST tests.  Also producing 
entirely different tests depending on  some property of the parser, 
just complicates matters and would create
four or more variants of one test that you would have to keep in synch.

Received on Monday, 23 April 2001 15:02:58 UTC