W3C home > Mailing lists > Public > public-rdf-dawg-comments@w3.org > January 2007

ACTION: SimonR to look at .Unit

From: Simon Raboczi <raboczi@itee.uq.edu.au>
Date: Tue, 2 Jan 2007 11:42:06 +1000
Message-Id: <0594497A-D89E-4CC3-A77D-F025AFCE8459@itee.uq.edu.au>
To: public-rdf-dawg-comments@w3.org

I took an action [1] at the last teleconference to investigate  
whether the xUnit Test Driven Development (TDD) community had a  
standard test reporting format that might be an alternative to EARL  
[2] for the SPARQL tests [3].  (Posted to the comments list for the  
administrative reason that I'm not formally on the WG.)

Although originally developed for Smalltalk, the Java version JUnit  
was the first version of the testing framework to break into the  
mainstream.  In JUnit the results of running a test suite can be  
reported in one of two formats: raw text suitable for human  
inspection, or an XML format suitable for build automation tools  
(e.g. the continuous integration framework CruiseControl [4]).  It's  
this XML report format which serves a similar role to EARL.

JUnit's popularity saw ports to just about any language in popular  
use [6], giving rise to the "xUnit" family.  As far as I can tell,  
however, JUnit's XML reporting format does not appear to have become  
a de-facto standard across the family.  There does however appear to  
a modicum of interest in standardizing, particularly among smaller  
language communities who stand to gain access to mainstream build  
automation tools (e.g. Python and Lua, see below) and mixed-language  
projects needing to integrate test reports from different languages.   
David Collados Polidura has apparently created a JUnit-compatible XML  
reporter for CPPUnit (the C++ version of xUnit) and an XML Schema for  
the format [8], and Marc-Elian Bégin has done so [7] for PyUnit (the  
Python version).  I also ran across Scott Jacobs expressing the  
desirability of standardizing the XML reporting for Lua's version [5].

The impression I get is that although there's a modest desire out  
there for a standard test reporting format, and JUnit's XML reports  
are the closest thing to a standard at the moment, there's neither a  
formal nor a strong de-facto standard established just yet.


Marc-Elian Bégin (the Python guy) [7]:
 >
 > Testing of complex software systems is a tough business,  
especially when
 > you start mixing languages and test frameworks. The xUnit framework,
 > from the original JUnit test framework developed for Java, is the
 > de-facto unit testing framework in open source development.  However,
 > the output format of the tests is not standard across the different
 > implementations of xUnit, which is now available for nearly all  
popular
 > languages (and also exotic ones). This makes it difficult for a  
build and
 > test system to generate an overall report of the tests. Meanwhile,  
Ant’s
 > dedicated “junit” tasks outputs a test report in a simple XML  
format. The
 > reports being XML, makes further processing (e.g. search and
 > transformation) simple and convenient.  If we adopt this format,  
it would
 > be interesting to have other xUnit implementations to generate their
 > report in the same format (i.e. generate XML Schema compliant  
reports).


Scott Jacobs (the Lua guy) [5]:
 >
 > Using the common xml format makes the results of your unit testing
 > available to tools like CruiseControl.  For precisely that reason  
I did
 > add xunit-style xml output to a unit-testing framework that I  
wrote for
 > another language.  At the time, I just used examples of output from
 > unit-test runs that I stumbled across via google.  Since then I  
have come
 > across this link [8] purporting to be a schema for JUnit's xml  
output.


[1] http://www.w3.org/2006/12/19-dawg-minutes.html#action10
[2] http://www.w3.org/TR/EARL10/
[3] http://www.w3.org/2001/sw/DataAccess/tests/
[4] http://cruisecontrol.sourceforge.net/
[5] http://lua-users.org/lists/lua-l/2005-07/msg00367.html
[6] http://en.wikipedia.org/wiki/List_of_unit_testing_frameworks
[7] http://meb.home.cern.ch/meb/xPyUnit.htm
[8] http://jra1mw.cvs.cern.ch:8180/cgi-bin/jra1mw.cgi/ 
org.glite.testing.unit/config/JUnitXSchema.xsd?rev=HEAD&content- 
type=text/vnd.viewcvs-markup
Received on Tuesday, 2 January 2007 01:42:42 GMT

This archive was generated by hypermail 2.2.0+W3C-0.50 : Tuesday, 8 January 2008 14:14:51 GMT