RE: WSDL Interop progress report

Oops sorry.  My memory failed me there...

> -----Original Message-----
> From: Chathura Herath [mailto:chathurah@gmail.com]
> Sent: Tuesday, July 11, 2006 8:31 PM
> To: Jonathan Marsh; www-ws-desc@w3.org
> Subject: Re: WSDL Interop progress report
> 
> Jonathan,
> Its Indiana University/Axis2 not Illinois.
> Chathura
> 
> On 7/11/06, Jonathan Marsh <jmarsh@microsoft.com> wrote:
> >
> >
> >
> >
> > The Interop event held July 5-8th in Toronto marked substantial
progress
> > towards are CR implementation goals.  Present were IBM (Arthur
Ryman,
> John
> > Kaputin, Lawrence Mandel), Canon (Youenn Fablet), University of
> > Illinois/Axis2 (Chathura Herath), W3C (Philippe Le Hegaret), and
myself.
> >
> >
> >
> > Test suite health:
> >
> >
> >
> > The specs (Core, Adjuncts) contain 226 machine-readable assertions.
> > Philippe crafted a report [1] to categorize and list the assertions,
> > correlate them to test cases, and thereby measure our progress
towards a
> > test suite that contains at least one test case violating each
> assertion.
> > In the process we more finely tuned the test metadata formats.  Our
> bucket
> > of bad documents grew from 15 to 60.
> >
> >
> >
> > Our bucket of good documents likewise grew, from 35 testcases to 64.
> The
> > test coverage report [2] shows that 42% of the constructs are
covered by
> two
> > or more testcases.  Or if you rather, 51% is covered by one or more
> > testcases.
> >
> >
> >
> > Summary:
> >
> > Validation test suite completion: 25 of 226 assertions covered: 11%
> > complete.
> > Language coverage completion: 55 of 108 constructs covered by at
least
> one
> > test case: 51% complete.
> > Language coverage completion: 46 of 108 constructs covered by at
least
> two
> > test cases: 42% complete.
> >
> >
> >
> > Implementation health:
> >
> >
> >
> > We developed an initial format for reporting validation results [3],
and
> > rolling those results up into a report [4].  The format includes a
> facility
> > to check not just for a validation error, but that the assertion
> violation
> > has been reported correctly.  Woden is the only implementation
providing
> > preliminary results at this point.
> >
> >
> >
> > The interchange format results [5] continue to grow with the test
suite.
> > Some major bugs were fixed in the Woden implementation, leading to
> better
> > results, but much more diagnosis needs to be done, and updated Canon
> results
> > need to be obtained.
> >
> >
> >
> > We performed some message exchanges between Apache client/server and
> Canon
> > client/server (including self-tests).  We obtained logs of these
> exchanges
> > which were apparently successful, but still have to develop a
mechanism
> to
> > evaluate the results and display the results.
> >
> >
> >
> > Next steps:
> >
> > Develop bad testcases covering remaining 201 assertions.
> > Develop good testcases covering the remaining 50-60 constructs.
> > Improve test coverage report to show percentage complete.
> > Generate and evaluate additional Woden validation results.
> > Generate and evaluate Canon validation results.
> > Complete validation result display.
> > Generate and evaluate additional Woden interchange format results as
new
> > test cases are added.
> > Generate and evaluate additional Canon interchange format results as
new
> > test cases are added.
> > Generate and evaluate additional wsdl-xslt interchange format
results as
> new
> > test cases are added.
> > Develop mechanism to evaluate message exchanges.
> > Develop dashboard providing convenient display of all indicators.
> >
> >
> >
> > Still lots to do!  But the implementers present believe that we may
be
> able
> > to complete the test suite by early September.  We discussed
favorably
> the
> > possibility of holding another interop event in late September or
early
> > October to attempt to complete the interop work and enable the specs
to
> move
> > forward to PR shortly after.
> >
> >
> >
> > [1]
> >
http://dev.w3.org/cvsweb/~checkout~/2002/ws/desc/test-suite/assertions-
> report.html?content-type=text/html;%20charset=utf-8
> >
> > [2]
> >
http://dev.w3.org/cvsweb/~checkout~/2002/ws/desc/test-suite/test-suite-
> coverage-summary.xml?content-type=text/xml
> >
> > [3]
> > http://dev.w3.org/cvsweb/~checkout~/2002/ws/desc/test-
> suite/results/Woden/validation-results.xml
> >
> > [4]
> > http://dev.w3.org/cvsweb/~checkout~/2002/ws/desc/test-
> suite/results/implementations.xml
> >
> > [5]
> > http://dev.w3.org/cvsweb/~checkout~/2002/ws/desc/test-
> suite/results/Overview.html
> >
> >
> >
> >  [  Jonathan Marsh  ][  jmarsh@microsoft.com  ][
> > http://auburnmarshes.spaces.msn.com  ]
> >
> >
> 
> 
> --
> Chathura Herath
> http://people.apache.org/~chathura/
> http://chathurah.blogspot.com/

Received on Wednesday, 12 July 2006 13:57:07 UTC