RE: A fix for the async case and the WS-I logs

I entirely agree that going the '4 message' approach is probably useful 
and I'm happy to go that way if the impact in terms of delay isn't too 
great.

There is a problem with the XSLT you suggest at that is it relies on there 
only ever being 2 'conversations' which in my experience is unlikely.
FYI the WSI tool defines a conversation as an HTTP connection.

I took a simplistic approach to this problem by adding the second step 
which matched the second message element for a given testcase number and 
set message="2".
This worked as the empty messages had been removed and the WS-I logs are 
in strictly chronological order. We could do something similar in the 4 
message case - 
either an expansion of my clumsy XSLT or something cleaner which 
effectively sets message=count(number of previous elements with the same 
testcase number)+1

David

P.S. I couldn't agree more about spending time making multiple runs of a 
testcase in one log work.

David Illsley
Web Services Development
IBM Hursley Park, SO21 2JN
+44 (0)1962 815049 (Int. 245049)
david.illsley@uk.ibm.com



"Jonathan Marsh" <jmarsh@microsoft.com> 
07/02/2006 01:43

To
David Illsley/UK/IBM@IBMGB, <public-ws-addressing-tests@w3.org>
cc

Subject
RE: A fix for the async case and the WS-I logs






There are unfortunately a couple of test cases that rely on the message 
number ? 1236 and 1237.  It?s not clear these tests can be run reliably 
against the variety of log formats we have.  (So far these tests have been 
giving constant green results.)
 
Yuk, what a mess.  This boils down to the same concepts exposed in the 
debate between the correspondence of WSDL and SOAP MEPs.  Some of our 
logs/stylesheets/testcases assume the SOAP MEP model ? a non-anonymous r/r 
consists of two SOAP one-way MEPs and therefore 4 HTTP messages.  Some of 
our materials assume the WSDL MEP model, which means  2 messages for r/r 
regardless.
 
We need to decide which is the right approach, as many things need to be 
fixed either way.  The simplest fix is as David suggests, to strip out 
extra messages and not be able to examine HTTP responses without SOAP 
envelopes.
 
But it does seem like useful capability to be able to add assertions about 
HTTP responses without soap envelopes.  But the fixes involved are pretty 
invasive.
 
The fix to the WS-I logs can probably be done most easily directly in 
ws-i.xsl by inserting this fragment in place (two times) of the existing 
attribute generation:
 
            <xsl:attribute name="message">
                        <xsl:choose>
                                    <xsl:when test="@type='request' and 
@conversationID='1'">1</xsl:when>
                                    <xsl:when test="@type='response' and 
@conversationID='1'">2</xsl:when>
                                    <xsl:when test="@type='request' and 
@conversationID='2'">3</xsl:when>
                                    <xsl:when test="@type='response' and 
@conversationID='2'">4</xsl:when>
                        </xsl:choose>
            </xsl:attribute>
 
Microsoft log generation would have to be updated too, if not to insert 
the missing messages, at least to insert the right message ID (2s would 
have to become 3s on non-anonymous messages).
 
Unfortunately the testcases document does not use the message numbers from 
the message exchange document.  We should probably replace <message 
from=?A? to=?B? ?> with something like <message order=?1? ?>
 
Mkschema.xsl would need corresponding changes, to match assertions up with 
the correct message id number.
 
Some of the assertions (1236,1237) would need to be fixed ? they rely on 
fixed message numbers.  We could add additional assertions such as David 
suggests.
 
This is complex enough that I didn?t want to proceed without everyone?s 
buyoff.
 
Alternatively we could strip messages and number the messages just in 
ws-i.xsl, and maybe even live with or make small fixes to the testcases 
document and mkschema.xsl.  But that loses the ability to examine (or at 
least count) the HTTP responses.
 
P.S. I would be very surprised if running multiple tests in a single log 
works reliably in any case.  At this point I?d be reluctant to invest the 
diligence necessary to verify that this works?
 
 

From: public-ws-addressing-tests-request@w3.org 
[mailto:public-ws-addressing-tests-request@w3.org] On Behalf Of David 
Illsley
Sent: Sunday, January 29, 2006 8:50 AM
To: public-ws-addressing-tests@w3.org
Subject: A fix for the async case and the WS-I logs
 

I like green... and here's how we (well just IBM at the moment) can get 
some more... 

The area of the WS-I logs that I think caused most problems were those 
tests which use non-anonymous reply/fault to addresses. In these cases the 
WS-I monitor consists of 2 request response logs for each of these tests. 
The problem is how to select the 2 relevant messages from the 4 to apply 
the assertions againt. I'm proposing a pretty simplistic solution, which 
has at least one general down side but which I'm happy to accept for logs 
submitted by IBM. 

The approach is twofold: 
        1. Remove all messageEntry elements from the WS-I log files (or 
more accurately don't copy them across) which have an empty messageContent 
element 
                In order to confirm that messages haven't been removed 
that should have assertions applied againt them, I suggest adding the 
following assertion to the first message of the aync tests: 
        2. When there are multiple message elements in the WS-Addressing 
log file format with the same testcase id, renumber the message value of 
the second one to 2. 
                This is where the downside comes in ... it doesn't allow 
runs of the same test in the same log. It also mandates that there is no 
re-ordering of the logs but right now that isn't a problem. 
                There are more advanced things we could do to pick out the 
second message as a reply e.g. examining the contents for an echoOut or 
Fault element or even using the WSAddressin 
                 RelatesTo value but that was beyond my XPath expertise 

        In order to confirm that this all works can I also suggest adding 
the following assertion to the request messages of the async tests: 
                <assert test=
"../following-sibling::log:message[@testcase=current()/../@testcase and 
@message='2']"/> 

No doubt Jonathan and Paul can come up with a better way of doing it if 
they have time but to keep things going I've got a very basic working 
solution going (sorry if my XSLT/XPath is horrendously inefficient). 
It consists of 2 changes: 

1, ws-i.xsl: 
        Replace 
                        <xsl:for-each select="//wsi-log303:messageEntry"> 
        with 
                        <xsl:for-each 
select="//wsi-log303:messageEntry[string-length(wsi-log303:messageContent/text())>0]"> 

And similarly for the other namespace 

2. Add yet another file to the pipeline for the WS-I logs: fix-asyc.xslt 
<?xml version="1.0" encoding="UTF-8"?> 
<xsl:stylesheet version="1.0" xmlns:xsl="
http://www.w3.org/1999/XSL/Transform" xmlns:log="
http://www.w3.org/2002/ws/addressing/logs/" xmlns:soapenv="
http://schemas.xmlsoap.org/soap/envelope/"> 
        <xsl:output method="xml" version="1.0" encoding="UTF-8" 
indent="yes"/> 
        <xsl:template match="* | comment()"> 
                <xsl:copy> 
                        <xsl:apply-templates select="@*"/> 
                        <xsl:apply-templates/> 
                </xsl:copy> 
        </xsl:template> 
        <xsl:template match="log:message"> 
                <xsl:copy> 
                        <xsl:apply-templates select="@*"/> 
                        <xsl:call-template 
name="testcase-message-renumber"> 
                        <xsl:with-param name="testcase" 
select="@testcase"/> 
                        </xsl:call-template> 
                        <xsl:apply-templates/> 
                </xsl:copy> 
        </xsl:template> 
        <xsl:template name="testcase-message-renumber"> 
                <xsl:param name="testcase"/> 
                <xsl:attribute name="message"> 
                <xsl:choose> 
                <xsl:when 
test="preceding-sibling::log:message[@testcase=$testcase]">2</xsl:when> 
                <xsl:otherwise>1</xsl:otherwise></xsl:choose> 
                </xsl:attribute> 
        </xsl:template> 
        <xsl:template match="@*"> 
                <xsl:copy-of select="."/> 
        </xsl:template> 
</xsl:stylesheet> 

This should make the IBM-IBM column from the event logs all green... and 
should be usable by the other participants using the WS-I tool if they're 
happy to accept the limitation I mentioned above. 
Paul, if this seems reasonable can you add these changes to the workflow, 
at least for the IBM case? 
Now for some attention to the new tests! 
Cheers, 
David 

David Illsley
Web Services Development
IBM Hursley Park, SO21 2JN
+44 (0)1962 815049 (Int. 245049)
david.illsley@uk.ibm.com