testing report status

Progress, of a sort.

I've been trying to make headway with our testing report framework
to which end I've taken the SOAP4r toolkit. The script:

http://www.w3.org/2002/ws/databinding/edcopy/toolkits/soap4r_ruby_20051204/

The genit.sh script manufactures a wsdl file, chopping out the
schema patterns which cause it to bail (which in some cases 
reveals issues with our examples, e.g. TargetNamespace 
is our fault, not SOAP4r's.

The script then substitutes /NotImplementedError/
with return statements. 

the perl hack, runit.sh, takes each of the SOAP 1.1
instance documents and posts them to the Service endpoint:

http://www.w3.org/2002/ws/databinding/edcopy/toolkits/runit.sh

So the output required to generate the report is:

A manifest XML file:
http://www.w3.org/2002/ws/databinding/edcopy/toolkits/soap4r_ruby_20051204/manifest.xml

A report of the SOAP/XML messages sent an recieved:
http://www.w3.org/2002/ws/databinding/edcopy/toolkits/soap4r_ruby_20051204/output.xml

and ideally a HTML file with annotated fragids for each pattern:
http://www.w3.org/2002/ws/databinding/edcopy/toolkits/soap4r_ruby_20051204/snippets.html

I've also managed to gain results from the .NET 2.0, Mono, Axis 1.4 
and XFire toolkits, though am hitting a few issues using the tools

The next stage is to bundle the tests into 
a tabular form, as we did for the WS-Addressing testsuite:

http://www.w3.org/2002/ws/addr/testsuite/report/

to do that we still need to compare input and output messages,
especially in the case where floating point numbers are involved

Paul

Received on Tuesday, 12 September 2006 14:13:00 UTC