W3C home > Mailing lists > Public > www-ws-desc@w3.org > November 2006

RE: Requested enhancement to assertion coverage report

From: Lawrence Mandel <lmandel@ca.ibm.com>
Date: Tue, 28 Nov 2006 01:03:37 -0500
To: "Jonathan Marsh" <jonathan@wso2.com>
Cc: www-ws-desc@w3.org
Message-ID: <OFD45DAC1D.67FBA159-ON85257234.00201DF5-85257234.0021497D@ca.ibm.com>
You're fast. My thanks once again.

Lawrence Mandel

Software Developer
IBM Rational Software
Phone: 905 - 413 - 3814   Fax: 905 - 413 - 4920
lmandel@ca.ibm.com



"Jonathan Marsh" <jonathan@wso2.com> 
11/28/2006 12:38 AM

To
Lawrence Mandel/Toronto/IBM@IBMCA, <www-ws-desc@w3.org>
cc

Subject
RE: Requested enhancement to assertion coverage report






OK, makes sense.  Checked it in?
 
http://dev.w3.org/cvsweb/~checkout~/2002/ws/desc/test-suite/Assertion-coverage.html
 
 
Jonathan Marsh - http://www.wso2.com - 
http://auburnmarshes.spaces.live.com
 

From: Lawrence Mandel [mailto:lmandel@ca.ibm.com] 
Sent: Monday, November 27, 2006 9:03 PM
To: Jonathan Marsh; www-ws-desc@w3.org
Subject: RE: Requested enhancement to assertion coverage report
 

I thought it would be useful to see if all of our bad test cases are 
simple and fail a single assertion or if there are many that fail multiple 
assertions. (Real world test cases that I've seen for WSDL 1.1 often fail 
multiple assertions.) I think it would also be good to see if there are 
certain assertions that always fail together to determine if any dependent 
assertions have been missed. 

Lawrence Mandel

Software Developer
IBM Rational Software
Phone: 905 - 413 - 3814   Fax: 905 - 413 - 4920
lmandel@ca.ibm.com 


"Jonathan Marsh" <jonathan@wso2.com> 
Sent by: www-ws-desc-request@w3.org 
11/27/2006 11:22 PM 


To
Lawrence Mandel/Toronto/IBM@IBMCA, <www-ws-desc@w3.org> 
cc
 
Subject
RE: Requested enhancement to assertion coverage report
 


 
 




How does that help?  I can see a roll-up of how complex good testcases 
are, but what?s the difference in practice between a document that 
violates one assertion and one that violates two? 
  
Jonathan Marsh - http://www.wso2.com - 
http://auburnmarshes.spaces.live.com 
  
 


From: www-ws-desc-request@w3.org [mailto:www-ws-desc-request@w3.org] On 
Behalf Of Lawrence Mandel
Sent: Thursday, November 16, 2006 11:47 PM
To: www-ws-desc@w3.org
Subject: Re: Requested enhancement to assertion coverage report 
  

One further request. (Knew I forgot one when I clicked send.) 

3. I'd like to see the complexity of the test documents in the test suite. 
I've focused on creating very simple tests that for the most part violate 
one assertion. I think it would be useful to understand the complexity of 
the documents in the suite by viewing how many violate 1 assertion, 2 
assertions and 3 or more assertions. I'm thinking of a section such as 

Complexity of Test Documents: 
# docs that violate 1 assertion: 80 
# docs that violate 2 assertions: 10 
# docs that violate 3 or more assertions: 2 

Thanks again, 

Lawrence Mandel

Software Developer
IBM Rational Software
Phone: 905 - 413 - 3814   Fax: 905 - 413 - 4920
lmandel@ca.ibm.com 

Lawrence Mandel/Toronto/IBM@IBMCA 
Sent by: www-ws-desc-request@w3.org 
11/17/2006 02:36 AM 
 


To
www-ws-desc@w3.org 
cc
 
Subject
Requested enhancement to assertion coverage report

 
 


 
 






Jonathan, 

First of all, nice work on the reports. 

I have a couple requests, assuming you can find the time. :) 

1. Given that you found a couple tests that referenced invalid assertion 
IDs, I'd like to request that a section be added to the assertion report 
that shows the tests that are referencing invalid assertion IDs. I think 
this will be useful as assertions are removed from the spec and when the 
assertions are renumbered. 

2. It would also be nice to know (this really is only a nice to have item 
on the importance scale) the coverage of part 1 and part 2 of the spec. 
I'd like to request that the numbers be included in the totals summary. 
For example, 

98/228 assertions (60/157 component, 20/50 document, 18/18 exchange, 0/3 
message). 80/106 for part 1, 18/122 for part 2. 

Thanks, 

Lawrence Mandel

Software Developer
IBM Rational Software
Phone: 905 - 413 - 3814   Fax: 905 - 413 - 4920
lmandel@ca.ibm.com 
Received on Tuesday, 28 November 2006 06:03:51 GMT

This archive was generated by hypermail 2.2.0+W3C-0.50 : Monday, 7 December 2009 10:58:42 GMT