Re: Test Suite Update from Arthur Ryman on 2005-02-18 (www-ws-desc@w3.org from February 2005)

From: Arthur Ryman <ryman@ca.ibm.com>
Date: Fri, 18 Feb 2005 17:15:56 -0500
To: Lawrence Mandel <lmandel@ca.ibm.com>
Cc: www-ws-desc@w3.org
Message-ID: <OF8D1540D0.46020562-ON85256FAC.0078512B-85256FAC.007A4F0E@ca.ibm.com>

Lawrence,

Thx for the quick response. Let me explain the rationale behind the Test
Suite.

The design of the Test Suite is that it will contain only schema-valid
documents since schema validators are readily available. We aren't trying
to test the accuracy of schema validators. We want to focus on those
semantics of WSDL that cannot be expressed in schema since implementations
will have to be developed for that.

Think of errors in the documents as a kind of noise. Ill-formed XML is
very noisy. Well-formed XML, that is not schema valid is less noisy.
Schema valid documents are fairly clean, but might still have some noise.
That's where we need to focus.

Schema validation is the "low pass filter" for WSDL validation, i.e. if
the document is not schema valid then the WSDL validator should reject it.
This is analogous to the schema validator requiring that the document be
well-formed. An ill-formed document is obviously broken, so we bother
spending time trying to make sense of it?

I'm cc the mailing list to provide an audit trail for the new patch.

Arthur Ryman,
Rational Desktop Tools Development

phone: +1-905-413-3077, TL 969-3077
assistant: +1-905-413-2411, TL 969-2411
fax: +1-905-413-4920, TL 969-4920
mobile: +1-416-939-5063, text: 4169395063@fido.ca
intranet: http://labweb.torolab.ibm.com/DRY6/

Lawrence Mandel/Toronto/IBM@IBMCA
Sent by: www-ws-desc-request@w3.org
02/18/2005 04:20 PM

To
Arthur Ryman/Toronto/IBM@IBMCA
cc
www-ws-desc@w3.org, www-ws-desc-request@w3.org
Subject
Re: Test Suite Update

Hi Arthur,

I intentionally created test cases that fail WSDL 2.0 schema validation.
These test cases are missing required attributes, have invalid attribute
values, or have elements in incorrect locations. Seems to me that a test
suite for WSDL 2.0 should cover all that WSDL 2.0 specifies not just all
that can't be specified by schema. In this case, only the good tests
should be expected to pass schema validation.

That said, I've created another patch that only contains the WSDL files
that pass schema validation. I've left the test numbers intact so these
tests can be compared with the ones I posted earlier. You can find the
patch below.

Lawrence Mandel

Software Developer
IBM Rational Software
Phone: 905 - 413 - 3814 Fax: 905 - 413 - 4920
lmandel@ca.ibm.com

Arthur Ryman/Toronto/IBM
02/18/2005 03:59 PM

To
Lawrence Mandel/Toronto/IBM@IBMCA
cc
www-ws-desc@w3.org, www-ws-desc-request@w3.org
Subject
Re: Test Suite UpdateLink

Lawrence,

Thx for the test cases. I ran the build script to validate them but got
the following errors. All test cases, including bad ones, must validate
correcty versus the WSDL XSD. Are you getting different results? Here's my
output:

Buildfile: D:\workspaces\Eclipse V3.1 M4\W3C\test-suite\build.xml
xsd-schema:
[xmlvalidate] 1 file(s) have been successfully validated.
wsdl-schema:
[xmlvalidate] 3 file(s) have been successfully validated.
good:
[xmlvalidate] 11 file(s) have been successfully validated.
bad:
[xmlvalidate]
/D:/workspaces/Eclipse%20V3.1%20M4/W3C/test-suite/documents/bad/Service-10B/Service.wsdl:26:32:
cvc-complex-type.4: Attribute 'binding' must appear on element
'wsdl:endpoint'.
[xmlvalidate] D:\workspaces\Eclipse V3.1
M4\W3C\test-suite\documents\bad\Service-10B\Service.wsdl is not a valid
XML document
[xmlvalidate]
/D:/workspaces/Eclipse%20V3.1%20M4/W3C/test-suite/documents/bad/Service-11B/Service.wsdl:27:23:
cvc-datatype-valid.1.2.1: '' is not a valid value for 'QName'.
[xmlvalidate]
/D:/workspaces/Eclipse%20V3.1%20M4/W3C/test-suite/documents/bad/Service-11B/Service.wsdl:27:23:
cvc-attribute.3: The value '' of attribute 'binding' on element
'wsdl:endpoint' is not valid with respect to its type, 'QName'.
[xmlvalidate] D:\workspaces\Eclipse V3.1
M4\W3C\test-suite\documents\bad\Service-11B\Service.wsdl is not a valid
XML document
[xmlvalidate]
/D:/workspaces/Eclipse%20V3.1%20M4/W3C/test-suite/documents/bad/Service-16B/Service.wsdl:28:27:
cvc-complex-type.2.4.a: Invalid content was found starting with element
'wsdl:documentation'. One of '{"http://www.w3.org/2004/08/wsdl":endpoint,
"http://www.w3.org/2004/08/wsdl":feature, "http://www.w3.org/2004/08/wsdl
":property, WC[##other:"http://www.w3.org/2004/08/wsdl"]}' is expected.
[xmlvalidate] D:\workspaces\Eclipse V3.1
M4\W3C\test-suite\documents\bad\Service-16B\Service.wsdl is not a valid
XML document
[xmlvalidate]
/D:/workspaces/Eclipse%20V3.1%20M4/W3C/test-suite/documents/bad/Service-3B/Service.wsdl:25:18:
cvc-complex-type.2.4.b: The content of element 'wsdl:service' is not
complete. One of '{"http://www.w3.org/2004/08/wsdl":documentation, "
http://www.w3.org/2004/08/wsdl":endpoint, "http://www.w3.org/2004/08/wsdl
":feature, "http://www.w3.org/2004/08/wsdl":property, WC[##other:"
http://www.w3.org/2004/08/wsdl"]}' is expected.
[xmlvalidate] D:\workspaces\Eclipse V3.1
M4\W3C\test-suite\documents\bad\Service-3B\Service.wsdl is not a valid XML
document
[xmlvalidate]
/D:/workspaces/Eclipse%20V3.1%20M4/W3C/test-suite/documents/bad/Service-4B/Service.wsdl:24:38:
cvc-complex-type.4: Attribute 'interface' must appear on element
'wsdl:service'.
[xmlvalidate] D:\workspaces\Eclipse V3.1
M4\W3C\test-suite\documents\bad\Service-4B\Service.wsdl is not a valid XML
document
[xmlvalidate]
/D:/workspaces/Eclipse%20V3.1%20M4/W3C/test-suite/documents/bad/Service-5B/Service.wsdl:24:51:
cvc-datatype-valid.1.2.1: '' is not a valid value for 'QName'.
[xmlvalidate]
/D:/workspaces/Eclipse%20V3.1%20M4/W3C/test-suite/documents/bad/Service-5B/Service.wsdl:24:51:
cvc-attribute.3: The value '' of attribute 'interface' on element
'wsdl:service' is not valid with respect to its type, 'QName'.
[xmlvalidate] D:\workspaces\Eclipse V3.1
M4\W3C\test-suite\documents\bad\Service-5B\Service.wsdl is not a valid XML
document
[xmlvalidate]
/D:/workspaces/Eclipse%20V3.1%20M4/W3C/test-suite/documents/bad/Service-6B/Service.wsdl:24:57:
cvc-datatype-valid.1.2.1: '' is not a valid value for 'NCName'.
[xmlvalidate]
/D:/workspaces/Eclipse%20V3.1%20M4/W3C/test-suite/documents/bad/Service-6B/Service.wsdl:24:57:
cvc-attribute.3: The value '' of attribute 'name' on element
'wsdl:service' is not valid with respect to its type, 'NCName'.
[xmlvalidate] D:\workspaces\Eclipse V3.1
M4\W3C\test-suite\documents\bad\Service-6B\Service.wsdl is not a valid XML
document
[xmlvalidate]
/D:/workspaces/Eclipse%20V3.1%20M4/W3C/test-suite/documents/bad/Service-8B/Service.wsdl:27:42:
cvc-datatype-valid.1.2.1: '' is not a valid value for 'NCName'.
[xmlvalidate]
/D:/workspaces/Eclipse%20V3.1%20M4/W3C/test-suite/documents/bad/Service-8B/Service.wsdl:27:42:
cvc-attribute.3: The value '' of attribute 'name' on element
'wsdl:endpoint' is not valid with respect to its type, 'NCName'.
[xmlvalidate] D:\workspaces\Eclipse V3.1
M4\W3C\test-suite\documents\bad\Service-8B\Service.wsdl is not a valid XML
document
[xmlvalidate]
/D:/workspaces/Eclipse%20V3.1%20M4/W3C/test-suite/documents/bad/Service-9B/Service.wsdl:26:42:
cvc-complex-type.4: Attribute 'name' must appear on element
'wsdl:endpoint'.
[xmlvalidate] D:\workspaces\Eclipse V3.1
M4\W3C\test-suite\documents\bad\Service-9B\Service.wsdl is not a valid XML
document
[xmlvalidate] 24 file(s) have been successfully validated.
main:
BUILD SUCCESSFUL
Total time: 6 seconds

Arthur Ryman,
Rational Desktop Tools Development

Lawrence Mandel/Toronto/IBM@IBMCA
Sent by: www-ws-desc-request@w3.org
02/16/2005 05:52 PM

To
www-ws-desc@w3.org
cc

Subject
Test Suite Update

I've created some more test cases for the WSDL 2.0 test suite. These test
cases show valid and invalid uses of the service and endpoint elements by
exercising different options with the service name and interface
attributes and documentation element, and the endpoint name, binding, and
address attributes, and documentation element.

<wsdl:service name="EchoService" interface="tns:EchoInterface">
<wsdl:documentation>
The Echo service.
</wsdl:documentation>
<wsdl:endpoint
name="SOAPEndpoint"
binding="tns:EchoSOAPBinding"
address="http://www.example.com/services/EchoService"/>
</wsdl:service>

For example, the interface attribute is specified as
1) interface="tns:EchoInterface"
2) interface="EchoInterface"
3) interface="tns:WrongEchoInterface"
4) interface=""
5) is omitted all together.

Similar tests are performed for the name attributes and the binding and
address attributes. The presence, absence, and position of the
documentation and endpoint elements are tested as well.

This patch contains 3 valid and 16 invalid tests.

Lawrence Mandel

Software Developer
IBM Rational Software
Phone: 905 - 413 - 3814 Fax: 905 - 413 - 4920
lmandel@ca.ibm.com[attachment "test-suite-patch.txt" deleted by Arthur
Ryman/Toronto/IBM]

Attachments

text/plain attachment: test-suite-patch-mod.txt

Received on Friday, 18 February 2005 22:16:30 UTC