RE: Component Model Results

Yes, though those numbers are an artifact of the comparison stylesheet.
I changed it to stop comparing an element and its children if the name
isn't what you expect, which reports more accurately the number of
failures of the first test at 6 rather than 384.  Those failures are the
four missing bits + two (redundant) warnings about the wrong number of
children elements.  An XML diff of those files which isn't so sensitive
to missing elements reveals 46 differences - the four missing bits plus
their effect on canonicalized xml:id values throughout the document.
Either way is sufficient to pinpoint the problem fairly quickly.

 

Looking through the results a little more, most of the problems are
differences in the available elementDeclarations and typeDefinitions (my
problem or Woden's TBD), and the lack of import support in my
stylesheet.

 

________________________________

From: Rogers, Tony [mailto:Tony.Rogers@ca.com] 
Sent: Thursday, April 27, 2006 9:04 PM
To: Jonathan Marsh; Arthur Ryman; Youenn Fablet
Cc: www-ws-desc@w3.org; www-ws-desc-request@w3.org
Subject: RE: Component Model Results

 

I'm pleased to see that there are some with 0 failures - that's looking
good. Not so good to see the ones with hundreds, though :-)

 

Tony 

________________________________

From: www-ws-desc-request@w3.org on behalf of Jonathan Marsh
Sent: Fri 28-Apr-06 11:42
To: Arthur Ryman; Youenn Fablet
Cc: www-ws-desc@w3.org; www-ws-desc-request@w3.org
Subject: RE: Component Model Results

I managed to run a pass between my stylesheet and Arthur's files.  None
of them matched perfectly of course, but I was able to fix my stylesheet
to get rid of many of the errors: I didn't put xml:id attributes on all
components - just those that needed referring to.  Also, I omitted the
build-in simple-type definitions.

 

I'm sure there are many more errors in my stylesheet, which doesn't even
attempt any wsdl:imports.  However, for the first file I looked at
carefully I think there's a Woden problem ;-)

 

My test result includes /descriptionComponent/elementDeclarations
(missing in Woden), with two components.  These are referred to from the
interfaceMessageReferenceComponents through the elementDeclaration
property (also missing in Woden).  My typeDefinitions element contains
two components (missing in Woden), the cinfoct and ccinfct types defined
in the included schema.

 

I also added a "folder" element to test-suite.xml to enable me to build
paths flexibly.  One of the id/foldernames didn't match the file system
so I fixed that.  It would be nice to add a "good/bad" attribute so I
don't have to string-search the id. ;-)  Checked in.

 

Lots of items on my task list now to clean up the results.  Here's the
raw numbers though (missing elements in the aspirant generate lots of
error messages.)  I'm encouraged by the three passing marks!

 


Results

documents/good/Chameleon-1G

getBalance.wsdl

Number of failures: 384

documents/good/Chameleon-2G

getBalance.wsdl

Number of failures: 55

documents/good/Chameleon-3G

getBalance.wsdl

Number of failures: 384

documents/good/Chameleon-4G

getBalance.wsdl

Number of failures: 384

documents/good/CreditCardFaults-1G

use-credit-card-faults.wsdl

Number of failures: 448

documents/good/GreatH-1G

primer-hotelReservationService.wsdl

Number of failures: 46

documents/good/Import-1G

XSDImport.wsdl

Number of failures: 66

documents/good/Import-2G

XSDImport.wsdl

Number of failures: 66

documents/good/Import-2G

XSDImport2.wsdl

Number of failures: 117

documents/good/ImportedWSDL-1G

retrieveDetails.wsdl

Number of failures: 4

documents/good/ImportedWSDL-1G

updateDetails.wsdl

Number of failures: 140

documents/good/Interface-1G

Interface.wsdl

Number of failures: 390

documents/good/Interface-2G

Interface.wsdl

Number of failures: 390

documents/good/Interface-3G

Interface.wsdl

Number of failures: 390

documents/good/Interface-4G

Interface.wsdl

Number of failures: 390

documents/good/Interface-5G

Interface.wsdl

Number of failures: 390

documents/good/Interface-6G

Interface.wsdl

Number of failures: 390

documents/good/Interface-7G

Interface.wsdl

Number of failures: 390

documents/good/MultipleInlineItems-1G

retrieveItems.wsdl

Number of failures: 11

documents/good/SchemaId-1G

schemaIds.wsdl

Number of failures: 54

documents/good/SchemaLocationFragment-1G

Items.wsdl

Number of failures: 3

documents/good/Service-1G

Service.wsdl

Number of failures: 0

documents/good/Service-2G

Service.wsdl

Number of failures: 0

documents/good/Service-3G

Service.wsdl

Number of failures: 0

documents/good/ServiceReference-1G

reservationDetails.wsdl

Number of failures: 162

documents/good/ServiceReference-1G

reservationList.wsdl

Number of failures: 194

documents/good/TicketAgent-1G

TicketAgent.wsdl

Number of failures: 88

documents/good/WeathSvc-1G

WeathSvc.wsdl

Number of failures: 149

documents/good/XsImport-1G

reservation.wsdl

Number of failures: 11

documents/good/XsImport-2G

reservationDetails.wsdl

Number of failures: 93

documents/good/XsImport-2G

reservationItems.wsdl

Number of failures: 3

documents/good/XsImport-3G

reservationDetails.wsdl

Number of failures: 93

documents/good/XsImport-3G

reservationItems.wsdl

Number of failures: 3

 

 

________________________________

From: www-ws-desc-request@w3.org [mailto:www-ws-desc-request@w3.org] On
Behalf Of Arthur Ryman
Sent: Thursday, April 27, 2006 9:10 AM
To: Youenn Fablet
Cc: www-ws-desc@w3.org; www-ws-desc-request@w3.org
Subject: Re: Component Model Results

 

***********************
Warning: Your file, documents.zip, contains more than 32 files after
decompression and cannot be scanned.
***********************


Youenn, 

I thought I fixed the xs:string problem which you previuosly reported.
I'll doublecheck. 

Here is my view on the purpose of the testing effort. At this point we
are really debugging the spec. It is reasonable for a given
implementation to fail some tests. We are not certifying
implementations. However, if no implementation can successfully
implement some feature, then we need to understand why, and potentially
alter the spec. 

Interoperability is one way to test the correctness of implementations
since it is unlikely that they will possess bugs that precisely cancel
eachother. Even if just one implementation implements a feature, we can
visually inspect it for correctness. 

Arthur Ryman,
IBM Software Group, Rational Division

blog: http://ryman.eclipsedevelopersjournal.com/
phone: +1-905-413-3077, TL 969-3077
assistant: +1-905-413-2411, TL 969-2411
fax: +1-905-413-4920, TL 969-4920
mobile: +1-416-939-5063, text: 4169395063@fido.ca 

Youenn Fablet <youenn.fablet@crf.canon.fr> 
Sent by: www-ws-desc-request@w3.org 

04/27/2006 11:49 AM 

To

www-ws-desc@w3.org 

cc

 

Subject

Component Model Results

 

 

 




***********************
Warning: Your file, documents.zip, contains more than 32 files after
decompression and cannot be scanned.
***********************


Hi all,
I have just finished generating the component model dumps of the wsdl 
good documents.
I just picked the test-suite.xml file and generated my test script 
through XSLT. Fairly quick in fact :-)
Please find attached the results.
Two side notes:
   - There is still a problem with 3 testsuite wsdl files 
(SchemaLocationFragment-1G/Items.wsdl, 
XSImport-2G/reservationItems.wsdl, XSImport-3G/reservationItems.wsdl). 
All of these files make reference to XSD data types through QNames but 
with a wrong namespace. I have fixed these files locally (type="string" 
is now type="xs:string") and produced a dump with these fixed files. I 
added the fixed wsdl files in the attached zip file.
   - Our implementation still fails on the Import-2G example as it 
raises an error when trying to create two components of the same type 
with the same ns+name, the reason being that we have not implemented the

component equivalence rules. Anyway, the parser still produces a 
component model dump.
I have a process question related to the last point: do we really need 
two interoperable implementations for every aspect of the spec, e.g. the

equivalence rules ? Or is it sufficient to document the reasons behind 
different behaviors ?
Hope this helps anyway,
Regards,
   Youenn