RE: Bug in test methodology.

I commented earlier that I wasn't too worried about this because it didn't 
affect the WS-I logs.. but it does for the async ones.
If the actual response comes on the HTTP response and there is no message 
3 then the message 3 assertions don't get tested.
This occurs today for the IBM->Sun log test 1251

David

David Illsley
Web Services Development
MP127, IBM Hursley Park, SO21 2JN
+44 (0)1962 815049 (Int. 245049)
david.illsley@uk.ibm.com



"Jonathan Marsh" <jmarsh@microsoft.com> 
Sent by: public-ws-addressing-tests-request@w3.org
24/02/2006 21:42

To
"Jonathan Marsh" <jmarsh@microsoft.com>, 
<public-ws-addressing-tests@w3.org>
cc

Subject
RE: Bug in test methodology.






This assertion which seemed to catch the problem for 1235: 
 
  ../following-sibling::log:message[@testcase=current()/../@testcase and 
@message!='1']/log:content/*
 

From: public-ws-addressing-tests-request@w3.org 
[mailto:public-ws-addressing-tests-request@w3.org] On Behalf Of Jonathan 
Marsh
Sent: Friday, February 24, 2006 10:07 AM
To: public-ws-addressing-tests@w3.org
Subject: Bug in test methodology.
 
I was running some rollups on the latest Microsoft results (hope to post 
those shortly!) and had a discrepancy between what the rollup told me and 
what the developers reported.  With test 1235, we swallowed the response 
message, so the log only contains a message=?1? message.  There is no 
message=?2?.  The assertions against message=?2? should fail, but the way 
processor-xmlout.xsl works if there is no message, there are no assertion 
failures to report.  That?s good in the case of testcases where there are 
no logs whatsoever, but isn?t so good in the case where one of the 
messages is simply dropped.
 
I don?t know the best way to fix this, perhaps adding in more assertions, 
1235 and in other cases, that check for a given message=?1? that there 
indeed exists a message=?2??
 
 [  Jonathan Marsh  ][  jmarsh@microsoft.com  ][  
http://spaces.msn.com/auburnmarshes  ]