W3C home > Mailing lists > Public > xmlschema-dev@w3.org > November 2006

Saxon-SA test results

From: Michael Kay <mike@saxonica.com>
Date: Wed, 15 Nov 2006 23:53:22 -0000
To: <xmlschema-dev@w3.org>
Message-ID: <01a801c70911$3bc811d0$6401a8c0@turtle>

First results for the new test suite for Saxon-SA

    expected    Saxon-SA     No of tests
    result      result

    valid       valid           24687
    invalid     invalid         13150
    valid       invalid           697
    invalid     valid             856

    not run                         9

That's 96.03% agreement with the expected results.

I'm sure it should be possible to raise that figure quite quickly - I
haven't started looking at the reasons for the discrepancies (with over 1500
of them, it's a bit daunting!). On past experience, a lot of the
discrepancies will turn out to be legitimate, e.g. because of limits that
are implementation-defined.

I'll submit the results file as soon as I've got it into shape.

The "not run" tests included 6 that blew memory because of large finite
state automata, plus 3 caused by the problem I raised a few minutes ago on
this list - user types that restrict from xs:anySimpleType.

These figures are after fixing about 8 new bugs in Saxon, most of them
crashing on error paths after reporting an error during schema analysis and
trying to carry on to find further errors.

Michael Kay
http://www.saxonica.com/


    
Received on Thursday, 16 November 2006 03:21:09 UTC

This archive was generated by hypermail 2.3.1 : Wednesday, 5 February 2014 07:15:11 UTC