- From: Jonathan Marsh <jmarsh@microsoft.com>
- Date: Fri, 17 Feb 2006 08:45:06 -0800
- To: "David Illsley" <david.illsley@uk.ibm.com>
- Cc: <public-ws-addressing-tests@w3.org>
- Message-ID: <37D0366A39A9044286B2783EB4C3C4E801A18F46@RED-MSG-10.redmond.corp.microsoft.com>
It looks like 4-message logs will be some amount of work, but there is a
halfway solution which I think we should pursue. Basically keep the
disjunctions in the XPaths so that missing response messages are treated
the same as empty response messages, but number the messages on a
4-message basis. So, MS would generate messages 1 and 3 for some of the
non-anonymous cases, others would generate 1,2,3,4. The XPaths would
check (as they do today) that messages 2 and 4 or either empty or not
logged.
Workable?
________________________________
From: David Illsley [mailto:david.illsley@uk.ibm.com]
Sent: Tuesday, February 07, 2006 9:18 AM
To: Jonathan Marsh
Cc: public-ws-addressing-tests@w3.org
Subject: RE: A fix for the async case and the WS-I logs
I entirely agree that going the '4 message' approach is probably useful
and I'm happy to go that way if the impact in terms of delay isn't too
great.
There is a problem with the XSLT you suggest at that is it relies on
there only ever being 2 'conversations' which in my experience is
unlikely.
FYI the WSI tool defines a conversation as an HTTP connection.
I took a simplistic approach to this problem by adding the second step
which matched the second message element for a given testcase number and
set message="2".
This worked as the empty messages had been removed and the WS-I logs are
in strictly chronological order. We could do something similar in the 4
message case -
either an expansion of my clumsy XSLT or something cleaner which
effectively sets message=count(number of previous elements with the same
testcase number)+1
David
P.S. I couldn't agree more about spending time making multiple runs of a
testcase in one log work.
David Illsley
Web Services Development
IBM Hursley Park, SO21 2JN
+44 (0)1962 815049 (Int. 245049)
david.illsley@uk.ibm.com
"Jonathan Marsh" <jmarsh@microsoft.com>
07/02/2006 01:43
To
David Illsley/UK/IBM@IBMGB, <public-ws-addressing-tests@w3.org>
cc
Subject
RE: A fix for the async case and the WS-I logs
There are unfortunately a couple of test cases that rely on the message
number - 1236 and 1237. It's not clear these tests can be run reliably
against the variety of log formats we have. (So far these tests have
been giving constant green results.)
Yuk, what a mess. This boils down to the same concepts exposed in the
debate between the correspondence of WSDL and SOAP MEPs. Some of our
logs/stylesheets/testcases assume the SOAP MEP model - a non-anonymous
r/r consists of two SOAP one-way MEPs and therefore 4 HTTP messages.
Some of our materials assume the WSDL MEP model, which means 2 messages
for r/r regardless.
We need to decide which is the right approach, as many things need to be
fixed either way. The simplest fix is as David suggests, to strip out
extra messages and not be able to examine HTTP responses without SOAP
envelopes.
But it does seem like useful capability to be able to add assertions
about HTTP responses without soap envelopes. But the fixes involved are
pretty invasive.
The fix to the WS-I logs can probably be done most easily directly in
ws-i.xsl by inserting this fragment in place (two times) of the existing
attribute generation:
<xsl:attribute name="message">
<xsl:choose>
<xsl:when test="@type='request' and
@conversationID='1'">1</xsl:when>
<xsl:when test="@type='response' and
@conversationID='1'">2</xsl:when>
<xsl:when test="@type='request' and
@conversationID='2'">3</xsl:when>
<xsl:when test="@type='response' and
@conversationID='2'">4</xsl:when>
</xsl:choose>
</xsl:attribute>
Microsoft log generation would have to be updated too, if not to insert
the missing messages, at least to insert the right message ID (2s would
have to become 3s on non-anonymous messages).
Unfortunately the testcases document does not use the message numbers
from the message exchange document. We should probably replace <message
from="A" to="B" ...> with something like <message order="1" ...>
Mkschema.xsl would need corresponding changes, to match assertions up
with the correct message id number.
Some of the assertions (1236,1237) would need to be fixed - they rely on
fixed message numbers. We could add additional assertions such as David
suggests.
This is complex enough that I didn't want to proceed without everyone's
buyoff.
Alternatively we could strip messages and number the messages just in
ws-i.xsl, and maybe even live with or make small fixes to the testcases
document and mkschema.xsl. But that loses the ability to examine (or at
least count) the HTTP responses.
P.S. I would be very surprised if running multiple tests in a single log
works reliably in any case. At this point I'd be reluctant to invest
the diligence necessary to verify that this works...
________________________________
From: public-ws-addressing-tests-request@w3.org
[mailto:public-ws-addressing-tests-request@w3.org] On Behalf Of David
Illsley
Sent: Sunday, January 29, 2006 8:50 AM
To: public-ws-addressing-tests@w3.org
Subject: A fix for the async case and the WS-I logs
I like green... and here's how we (well just IBM at the moment) can get
some more...
The area of the WS-I logs that I think caused most problems were those
tests which use non-anonymous reply/fault to addresses. In these cases
the WS-I monitor consists of 2 request response logs for each of these
tests.
The problem is how to select the 2 relevant messages from the 4 to apply
the assertions againt. I'm proposing a pretty simplistic solution, which
has at least one general down side but which I'm happy to accept for
logs submitted by IBM.
The approach is twofold:
1. Remove all messageEntry elements from the WS-I log files (or
more accurately don't copy them across) which have an empty
messageContent element
In order to confirm that messages haven't been removed
that should have assertions applied againt them, I suggest adding the
following assertion to the first message of the aync tests:
2. When there are multiple message elements in the WS-Addressing
log file format with the same testcase id, renumber the message value of
the second one to 2.
This is where the downside comes in ... it doesn't allow
runs of the same test in the same log. It also mandates that there is no
re-ordering of the logs but right now that isn't a problem.
There are more advanced things we could do to pick out
the second message as a reply e.g. examining the contents for an echoOut
or Fault element or even using the WSAddressin
RelatesTo value but that was beyond my XPath expertise
In order to confirm that this all works can I also suggest adding
the following assertion to the request messages of the async tests:
<assert
test="../following-sibling::log:message[@testcase=current()/../@testcase
and @message='2']"/>
No doubt Jonathan and Paul can come up with a better way of doing it if
they have time but to keep things going I've got a very basic working
solution going (sorry if my XSLT/XPath is horrendously inefficient).
It consists of 2 changes:
1, ws-i.xsl:
Replace
<xsl:for-each select="//wsi-log303:messageEntry">
with
<xsl:for-each
select="//wsi-log303:messageEntry[string-length(wsi-log303:messageConten
t/text())>0]">
And similarly for the other namespace
2. Add yet another file to the pipeline for the WS-I logs: fix-asyc.xslt
<?xml version="1.0" encoding="UTF-8"?>
<xsl:stylesheet version="1.0"
xmlns:xsl="http://www.w3.org/1999/XSL/Transform"
xmlns:log="http://www.w3.org/2002/ws/addressing/logs/"
xmlns:soapenv="http://schemas.xmlsoap.org/soap/envelope/">
<xsl:output method="xml" version="1.0" encoding="UTF-8"
indent="yes"/>
<xsl:template match="* | comment()">
<xsl:copy>
<xsl:apply-templates select="@*"/>
<xsl:apply-templates/>
</xsl:copy>
</xsl:template>
<xsl:template match="log:message">
<xsl:copy>
<xsl:apply-templates select="@*"/>
<xsl:call-template
name="testcase-message-renumber">
<xsl:with-param name="testcase"
select="@testcase"/>
</xsl:call-template>
<xsl:apply-templates/>
</xsl:copy>
</xsl:template>
<xsl:template name="testcase-message-renumber">
<xsl:param name="testcase"/>
<xsl:attribute name="message">
<xsl:choose>
<xsl:when
test="preceding-sibling::log:message[@testcase=$testcase]">2</xsl:when>
<xsl:otherwise>1</xsl:otherwise></xsl:choose>
</xsl:attribute>
</xsl:template>
<xsl:template match="@*">
<xsl:copy-of select="."/>
</xsl:template>
</xsl:stylesheet>
This should make the IBM-IBM column from the event logs all green... and
should be usable by the other participants using the WS-I tool if
they're happy to accept the limitation I mentioned above.
Paul, if this seems reasonable can you add these changes to the
workflow, at least for the IBM case?
Now for some attention to the new tests!
Cheers,
David
David Illsley
Web Services Development
IBM Hursley Park, SO21 2JN
+44 (0)1962 815049 (Int. 245049)
david.illsley@uk.ibm.com
Received on Friday, 17 February 2006 16:45:26 UTC