RE: Validation results

Lawrence,

It would be very difficult to list the dependency information of each 
assertion, although I think that would be useful.

What we need here is essentially the preconditions for each assertion. A 
validator should not bother to test an assertion if any of its 
preconditions fail. So this is really not something to add to the test 
suite, but rather to the spec itself. Each assertion in the spec should 
have a list of precondition assertions. 

I suppose we could do this incrementally to start, i.e. when we find an 
assertion that depends on others being satisfied, then list them as 
preconditions. This information would be provided to validators in order 
to eliminate spurious violation reports.

Arthur Ryman,
IBM Software Group, Rational Division

blog: http://ryman.eclipsedevelopersjournal.com/
phone: +1-905-413-3077, TL 969-3077
assistant: +1-905-413-2411, TL 969-2411
fax: +1-905-413-4920, TL 969-4920
mobile: +1-416-939-5063, text: 4169395063@fido.ca



Lawrence Mandel/Toronto/IBM@IBMCA 
Sent by: www-ws-desc-request@w3.org
12/05/2006 08:30 AM

To
Arthur Ryman/Toronto/IBM@IBMCA
cc
"Jonathan Marsh" <jonathan@wso2.com>, "'www-ws-desc'" 
<www-ws-desc@w3.org>, www-ws-desc-request@w3.org
Subject
RE: Validation results







>It may report others since in some cases an assertion doesn't make sense 
unless other assertions are satisfied. 

In this case, where multiple assertions are violated, I think the test 
case should list all of the violated assertions. If this is the case the 
test should not show green unless the implementation reports all the 
assertion violations and only the assertion violations listed in the test 
case. 

Lawrence Mandel

Software Developer
IBM Rational Software
Phone: 905 - 413 - 3814   Fax: 905 - 413 - 4920
lmandel@ca.ibm.com 


Arthur Ryman/Toronto/IBM 
12/04/2006 06:13 PM 


To
"Jonathan Marsh" <jonathan@wso2.com> 
cc
Lawrence Mandel/Toronto/IBM@IBMCA, "'www-ws-desc'" <www-ws-desc@w3.org>, 
www-ws-desc-request@w3.org 
Subject
RE: Validation resultsLink







Jonathan, 

I agree that if the processor reports the expected assertion that it gets 
a green. It may report others since in some cases an assertion doesn't 
make sense unless other assertions are satisfied. 

Ideally, a processor should limit the number of assertion violations 
reported to eliminate ones that are consequences of errors. 

Arthur Ryman,
IBM Software Group, Rational Division

blog: http://ryman.eclipsedevelopersjournal.com/
phone: +1-905-413-3077, TL 969-3077
assistant: +1-905-413-2411, TL 969-2411
fax: +1-905-413-4920, TL 969-4920
mobile: +1-416-939-5063, text: 4169395063@fido.ca 


"Jonathan Marsh" <jonathan@wso2.com> 
Sent by: www-ws-desc-request@w3.org 
12/04/2006 05:21 PM 


To
Lawrence Mandel/Toronto/IBM@IBMCA 
cc
"'www-ws-desc'" <www-ws-desc@w3.org> 
Subject
RE: Validation results








Yes, so far it?s by design.  Validation is considered successful if, among 
the identified assertions that have been violated is the one associated 
with the test case.  It?s not considered unsuccessful if it identifies 
additional assertions that have been violated ? or multiple instances of 
violating a single assertion. 
  
However, I put that in place without much thought.  It would not be hard 
to flag those results that weren?t one-to-one matches with the identified 
assertion.  Whatever makes most sense to Woden at this point? 
  
Arthur, what do you think? 
  
Jonathan Marsh - http://www.wso2.com - 
http://auburnmarshes.spaces.live.com 
  



From: Lawrence Mandel [mailto:lmandel@ca.ibm.com] 
Sent: Monday, December 04, 2006 12:11 PM
To: Jonathan Marsh
Cc: 'www-ws-desc'
Subject: RE: Validation results 
  

Jonathan, 

With your most recent change I see all green on tests such as Binding-3B, 
even though the test cases indicates that only assertion Binding-0057 
should be violated and Woden identifies that assertions Binding-0057 and 
Binding-0055 have been violated. Is this by design? I would think 
assertion Binding-0055 should be flagged with a yellow or red marker. 

Lawrence Mandel

Software Developer
IBM Rational Software
Phone: 905 - 413 - 3814   Fax: 905 - 413 - 4920
lmandel@ca.ibm.com 

"Jonathan Marsh" <jonathan@wso2.com> 
Sent by: www-ws-desc-request@w3.org 
12/04/2006 01:29 PM 


To
"'Youenn Fablet'" <youenn.fablet@crf.canon.fr> 
cc
"'www-ws-desc'" <www-ws-desc@w3.org> 
Subject
RE: Validation results

  


  
 






Good catch, I've fixed it, with more green appearing as a result.  Checked
in your results too:

http://dev.w3.org/cvsweb/~checkout~/2002/ws/desc/test-suite/results/Validati

on.html

Jonathan Marsh - http://www.wso2.com - 
http://auburnmarshes.spaces.live.com


> -----Original Message-----
> From: Youenn Fablet [mailto:youenn.fablet@crf.canon.fr]
> Sent: Monday, December 04, 2006 8:39 AM
> To: Jonathan Marsh; www-ws-desc
> Subject: Validation results
> 
> I have regenerated the validation results.
> I have noted that the report is not quite accurate with the good
> documents.
> They are either flagged red or yellow, while they should be either red
> or green.
> Maybe a small change in the presentation stylesheet would greatly help
> improving the overall results ;-)
> Jonathan, can you update the CVS canon validation results file with this
> one?
> Thanks,
>     Youenn
>