W3C home > Mailing lists > Public > xmlschema-dev@w3.org > November 2006

Saxon-SA test results

From: Michael Kay <mike@saxonica.com>
Date: Wed, 15 Nov 2006 23:53:22 -0000
To: <xmlschema-dev@w3.org>
Message-ID: <01a801c70911$3bc811d0$6401a8c0@turtle>

First results for the new test suite for Saxon-SA

    expected    Saxon-SA     No of tests
    result      result

    valid       valid           24687
    invalid     invalid         13150
    valid       invalid           697
    invalid     valid             856

    not run                         9

That's 96.03% agreement with the expected results.

I'm sure it should be possible to raise that figure quite quickly - I
haven't started looking at the reasons for the discrepancies (with over 1500
of them, it's a bit daunting!). On past experience, a lot of the
discrepancies will turn out to be legitimate, e.g. because of limits that
are implementation-defined.

I'll submit the results file as soon as I've got it into shape.

The "not run" tests included 6 that blew memory because of large finite
state automata, plus 3 caused by the problem I raised a few minutes ago on
this list - user types that restrict from xs:anySimpleType.

These figures are after fixing about 8 new bugs in Saxon, most of them
crashing on error paths after reporting an error during schema analysis and
trying to carry on to find further errors.

Michael Kay

Received on Thursday, 16 November 2006 03:21:09 UTC

This archive was generated by hypermail 2.3.1 : Wednesday, 7 January 2015 14:56:11 UTC