RE: Validation results

>It may report others since in some cases an assertion doesn't make sense 
unless other assertions are satisfied.

In this case, where multiple assertions are violated, I think the test 
case should list all of the violated assertions. If this is the case the 
test should not show green unless the implementation reports all the 
assertion violations and only the assertion violations listed in the test 
case.

Lawrence Mandel

Software Developer
IBM Rational Software
Phone: 905 - 413 - 3814   Fax: 905 - 413 - 4920
lmandel@ca.ibm.com



Arthur Ryman/Toronto/IBM
12/04/2006 06:13 PM

To
"Jonathan Marsh" <jonathan@wso2.com>
cc
Lawrence Mandel/Toronto/IBM@IBMCA, "'www-ws-desc'" <www-ws-desc@w3.org>, 
www-ws-desc-request@w3.org
Subject
RE: Validation results





Jonathan,

I agree that if the processor reports the expected assertion that it gets 
a green. It may report others since in some cases an assertion doesn't 
make sense unless other assertions are satisfied.

Ideally, a processor should limit the number of assertion violations 
reported to eliminate ones that are consequences of errors.

Arthur Ryman,
IBM Software Group, Rational Division

blog: http://ryman.eclipsedevelopersjournal.com/
phone: +1-905-413-3077, TL 969-3077
assistant: +1-905-413-2411, TL 969-2411
fax: +1-905-413-4920, TL 969-4920
mobile: +1-416-939-5063, text: 4169395063@fido.ca



"Jonathan Marsh" <jonathan@wso2.com> 
Sent by: www-ws-desc-request@w3.org
12/04/2006 05:21 PM

To
Lawrence Mandel/Toronto/IBM@IBMCA
cc
"'www-ws-desc'" <www-ws-desc@w3.org>
Subject
RE: Validation results






Yes, so far it?s by design.  Validation is considered successful if, among 
the identified assertions that have been violated is the one associated 
with the test case.  It?s not considered unsuccessful if it identifies 
additional assertions that have been violated ? or multiple instances of 
violating a single assertion.
 
However, I put that in place without much thought.  It would not be hard 
to flag those results that weren?t one-to-one matches with the identified 
assertion.  Whatever makes most sense to Woden at this point?
 
Arthur, what do you think?
 
Jonathan Marsh - http://www.wso2.com - 
http://auburnmarshes.spaces.live.com
 

From: Lawrence Mandel [mailto:lmandel@ca.ibm.com] 
Sent: Monday, December 04, 2006 12:11 PM
To: Jonathan Marsh
Cc: 'www-ws-desc'
Subject: RE: Validation results
 

Jonathan, 

With your most recent change I see all green on tests such as Binding-3B, 
even though the test cases indicates that only assertion Binding-0057 
should be violated and Woden identifies that assertions Binding-0057 and 
Binding-0055 have been violated. Is this by design? I would think 
assertion Binding-0055 should be flagged with a yellow or red marker. 

Lawrence Mandel

Software Developer
IBM Rational Software
Phone: 905 - 413 - 3814   Fax: 905 - 413 - 4920
lmandel@ca.ibm.com 


"Jonathan Marsh" <jonathan@wso2.com> 
Sent by: www-ws-desc-request@w3.org 
12/04/2006 01:29 PM 


To
"'Youenn Fablet'" <youenn.fablet@crf.canon.fr> 
cc
"'www-ws-desc'" <www-ws-desc@w3.org> 
Subject
RE: Validation results
 


 
 





Good catch, I've fixed it, with more green appearing as a result.  Checked
in your results too:

http://dev.w3.org/cvsweb/~checkout~/2002/ws/desc/test-suite/results/Validati

on.html

Jonathan Marsh - http://www.wso2.com - 
http://auburnmarshes.spaces.live.com


> -----Original Message-----
> From: Youenn Fablet [mailto:youenn.fablet@crf.canon.fr]
> Sent: Monday, December 04, 2006 8:39 AM
> To: Jonathan Marsh; www-ws-desc
> Subject: Validation results
> 
> I have regenerated the validation results.
> I have noted that the report is not quite accurate with the good
> documents.
> They are either flagged red or yellow, while they should be either red
> or green.
> Maybe a small change in the presentation stylesheet would greatly help
> improving the overall results ;-)
> Jonathan, can you update the CVS canon validation results file with this
> one?
> Thanks,
>     Youenn
>