SV: [General] Language-independent test representation

(Answering the latest mail in the thread)

We discussed the issue of representing tests in a language-neutral way
during this week's DOM face to face meeting. We have reached the following
conclusions:

1. The tests will be represented in a language-neutral, XML format. This
format will be posted to this list to be discussed (rationale: this
simplifies generating other bindings (both code directly and porting to any
of the xUnit frameworks)).
2. The DOM Test Suite will consist of this collection of tests plus XSLT
documents for each binding (be it to language or to xUnit framework)
desired. Developers are warmly encouraged to submit these stylesheets.
3. The XML format will eb published, toghether with a tutorial to help
people develop tests, on the DOM TS webpage.
4. The DOM TS will consist of these XML-encoded tests, the XSLT documents,
plus the Java and ECMAScript bindings generated from the XML source (as with
the DOM Specification documents).
5. We should make sure that we can express multiple assertions. This will be
included in the DTD.

Please note that we intend to support all major existing testing frameworks,
but want to be able to allow for organisations to write their own harnesses
around our tests. 

Also, the W3C DOM TS Process document indicates [1] that the license under
which the TS will be published is the W3C Document License.

I hope this covers all points that have been put forward in this thread. If
i have missed any, please reply indicating what I've missed.

Regards,

/Dimitris           

[1]: http://www.w3.org/2001/03/DOMConformanceTS-Process-20010316#ownership

-----Ursprungligt meddelande-----
Från: Arnold, Curt [mailto:Curt.Arnold@hyprotech.com]
Skickat: den 24 april 2001 21:59
Till: 'www-dom-ts@w3.org'
Ämne: RE: [General] Language-independent test representation




> -----Original Message-----
> From: Mary Brady [mailto:mbrady@nist.gov]
> Sent: Tuesday, April 24, 2001 2:05 PM
> To: Arnold, Curt; www-dom-ts@w3.org
> Subject: Re: [General] Language-independent test representation
> 
> 
> Hi Curt,
> 
> I've looked over some of the junit and jsunit tests and had 
> some questions.
> It appears that you have introduced some intermediate methods, such as
> getElementsByTagNameAndIndex, etc -- I'm curious as to why you did
> this.  As it turns out, we were just considering moving away from this
> approach in the NIST ECMAScript tests.  Although at one time I thought
> it better to have common methods/functions for tasks that 
> were repeated
> often, it has in the end made our ECMAScript tests much 
> harder to follow.
> The feedback that we have received seems to favor just using 
> the standard
> DOM calls, instead of having indirection inside the tests themselves.

The primary motivation was to encapsulate places where the language bindings
were substantially different.  For example, the base class as an
"int getLength(Node node)" that can be used to hide the differences between
the Java binding which would have been "node.getLength()" and the JavaScript
which would have been "node.length".  However, getElementByTagNameAndIndex
is really more of a convienience function.

> Am I right in assuming that you went through and hand-coded 
> all three of
> these solutions as opposed to any automated mechanisms?  I'm trying to
> understand the level of effort involved in converting what we already
> have --   We already have another 200 tests that
> have not yet been released that address namespace issues and 
> DOMExceptions.
> We also have the 600 HTML tests that have to be considered.

I'm not sure if you are talking about the conversion from the initial
NIST tests to a JUnit form or from JUnit to JSUnit and CppUnit.  The
conversion of the NIST tests to JUnit was pretty much a porting project,
however I was able to use regex replacement to get the source in pretty
good condition before manually going through each case.  It required me 
about 2 days to convert the existing 284 test cases.

The conversion of JUnit tests to JSUnit and CppUnit is highly automatable, 
though the first pass was done using a series of regex replacements and
iteration.


> Have you used any of the graphical interfaces to run the 
> tests?  If so, do
> you have any comments?
> 

Not extensively.  

The CppUnit graphical interface does not allow dynamic loading
of tests and is platform-specific (MFC), so I didn't really bother with it,
preferring 
to concentrate on the command line driver.  

The JSUnit tests only run within a browser, 
basically you browse for an HTML file containing the tests and press run, a
bar shows 
test progress and turns red if there are any failures or errors.  If you
double click
on any error or failure, you get a description.  I haven't tested the
JavaScript
tests since the release of JSUnit 1.2 which primarily added support for
passing a
description as part of the assertion at my suggestion.  The current JSUnit
code doesn't
take advantage of that new capability.

I haven't used the JUnit user interfaces, however I assume that they are
pretty good.


> I can't seem to find the JSUNIT download -- I can browse the 
> repository
> and look at the source for the tests, but can't seem to find 
> the app area --
> I wanted to try this out in Mozilla.

You need to download JSUnit 1.2 from http://www.edwardh.com/jsunit/ and
extract 
it into the domunit/jsunit directory (giving you a domunit/jsunit/jsunit/app
directory).  
JSUnit is suppose to work with Mozilla provided you implement a method to
return 
the names of all the test scripts (which I attempted to implement, but
haven't tested).
You will need to modify DOMTestCaseConfig.js to appropriately invoke the
Mozilla parser.

I'd also like to be able to test the Adobe SVG Viewer using the framework,
but I haven't
spent anytime figuring out how to do that without messing up the
modularization.

Received on Friday, 27 April 2001 12:18:06 UTC