RDF Parser Test cases report for Raptor

Taking the manifest and tests from:
  http://www.w3.org/2000/10/rdf-tests/rdfcore/Manifest.rdf

and running against the latest CVS version of my RDF/XML parser,
Raptor (there are still some bugs in it from these tests, they are
noted below).


Ignored Obsolete Tests:
   URL http://www.w3.org/2000/10/rdf-tests/rdfcore/rdf-containers-syntax-vs-schema/Manifest.rdf#test005

Ignoring Miscellaneous Tests:
   URL http://www.w3.org/2000/10/rdf-tests/rdfcore/rdfms-uri-substructure/Manifest.rdf#error001

Parser tests found:
  Positive: 156
  Negative: 57
APPROVED parser tests found:
  Positive: 133
  Negative: 29


Positive Test http://www.w3.org/2000/10/rdf-tests/rdfcore/rdf-ns-prefix-confusion/Manifest.rdf#test0007 failed
Positive Test http://www.w3.org/2000/10/rdf-tests/rdfcore/rdf-ns-prefix-confusion/Manifest.rdf#test0008 FAILED - N-Triples match failed

These are obsolete - they contain aboutEach / aboutEachPrefix in them.
Please delete them.


I'm still working on rdf:bagID implementation in my parser,
however I currently get a failure on:

  Positive Test http://www.w3.org/2000/10/rdf-tests/rdfcore/rdfms-nested-bagIDs/Manifest.rdf#test007  FAILED

  Error - URI http://www.w3.org/2000/10/rdf-tests/rdfcore/rdfms-nested-bagIDs/test007.rdf:25 - rdf:bagID is forbidden on property element prop with an rdf:resource or rdf:parseType attribute.

and that is correct from my reading of the syntax, you can't this:
    <some:prop rdf:resource="rel#uriRef" rdf:bagID="bag" some:propAttr="val"/>
so this test case is in error.


Positive Test http://www.w3.org/2000/10/rdf-tests/rdfcore/rdfms-nested-bagIDs/Manifest.rdf#test010b FAILED - N-Triples match failed
Positive Test http://www.w3.org/2000/10/rdf-tests/rdfcore/rdfms-nested-bagIDs/Manifest.rdf#test011a FAILED - N-Triples match failed
Positive Test http://www.w3.org/2000/10/rdf-tests/rdfcore/rdfms-nested-bagIDs/Manifest.rdf#test012b FAILED - N-Triples match failed

These fail due to the way the tests are encoded in the manifest.
test01a, 011b and 012a work correctly, since they are the alternate
n-triples results for the given rdf/xml.  I would rather that the
manifest listed a positive parser test with 1 rdf/xml file and 1 or
more n-triples results; rather than have multiple tests.


Positive Test http://www.w3.org/2000/10/rdf-tests/rdfcore/rdfms-not-id-and-resource-attr/Manifest.rdf#test003 FAILED - N-Triples match failed

This is due to the value of an empty literal property element.
We have decided that
  <ex:prop></ex:prop>  or it's xml equivalent <ex:prop/>
gives an empty literal string result.  This test assumes that
it gives a bnode.  (Aside: ARP also follows this error test case).

Please fix the n-triples:
  -_:j88096 <http://example.org/prop1> _:j88097 .
  +_:j88096 <http://example.org/prop1> "" .


See http://www.w3.org/TR/rdf-syntax-grammar/#emptyPropertyElt
first bullet point.


Positive Test http://www.w3.org/2000/10/rdf-tests/rdfcore/rdfms-xml-literal-namespaces/Manifest.rdf#test001 FAILED - N-Triples match failed
Positive Test http://www.w3.org/2000/10/rdf-tests/rdfcore/rdfms-xml-literal-namespaces/Manifest.rdf#test002 FAILED - N-Triples match failed

These are XML canonicalization errors probably on my part.  This is
going to be very hard for people to check.  No error.


Positive Test http://www.w3.org/2000/10/rdf-tests/rdfcore/xmlbase/Manifest.rdf#test005 FAILED - N-Triples match failed

  Error - URI http://www.w3.org/2000/10/rdf-tests/rdfcore/xmlbase/test005.rdf:19 - Document element rdf:RDF missing.

This is not RDF/XML, please obsolete this test.



Results for Positive Parser Tests
BAD       10    6.41%
OK       146   93.58%
TOTAL    156  100.00%

(noting that there are really 3 tests overlapped, the ones with
alternate answers)


For the negative tests, I'm still working on checking them.  I
currently have these unchecked cases:

Negative Test http://www.w3.org/2000/10/rdf-tests/rdfcore/rdf-charmod-literals/Manifest.rdf#error001 SUCCEEDED, should have failed
Negative Test http://www.w3.org/2000/10/rdf-tests/rdfcore/rdf-charmod-literals/Manifest.rdf#error002 SUCCEEDED, should have failed
Negative Test http://www.w3.org/2000/10/rdf-tests/rdfcore/rdf-charmod-uris/Manifest.rdf#error001 SUCCEEDED, should have failed

These are charmod-or IRI-related tests, but we are no long normative
on either.  These should be reviewed.  What are the normalization
requirements on something transforming rdf/xml to an rdf graph?  The
concepts doc mentions NFC, but the XML syntax doc never does. 

It seems to require an immense amount of code to support (even the
Java Jar file is megabytes) and looks unlikely I'll be getting round
to it anytime soon.


Negative Test http://www.w3.org/2000/10/rdf-tests/rdfcore/rdf-containers-syntax-vs-schema/Manifest.rdf#error001 SUCCEEDED, should have failed

My bug.


Negative Test http://www.w3.org/2000/10/rdf-tests/rdfcore/rdf-ns-prefix-confusion/Manifest.rdf#error0001 SUCCEEDED, should have failed

This uses a bare 'about' attribute.  This is no longer an error,
please obsolete and/or convert to a positive test case. 


Negative Test http://www.w3.org/2000/10/rdf-tests/rdfcore/rdf-ns-prefix-confusion/Manifest.rdf#error0002 SUCCEEDED, should have failed

This uses a bare 'bagID' attribute.  This is no longer an error,
please obsolete and/or convert to a positive test case. 


Negative Test http://www.w3.org/2000/10/rdf-tests/rdfcore/rdf-ns-prefix-confusion/Manifest.rdf#error0003 SUCCEEDED, should have failed

This uses a bare 'resource' attribute.  This is no longer an error,
please obsolete and/or convert to a positive test case. 


Negative Test http://www.w3.org/2000/10/rdf-tests/rdfcore/rdf-ns-prefix-confusion/Manifest.rdf#error0004 SUCCEEDED, should have failed

This uses a bare 'ID' attribute.  This is no longer an error,
please obsolete and/or convert to a positive test case. 


Negative Test http://www.w3.org/2000/10/rdf-tests/rdfcore/rdf-ns-prefix-confusion/Manifest.rdf#error0005 SUCCEEDED, should have failed

This uses a bare 'parseType' attribute.  This is no longer an error,
please obsolete and/or convert to a positive test case. 


Negative Test http://www.w3.org/2000/10/rdf-tests/rdfcore/rdfms-empty-property-elements/Manifest.rdf#error001 SUCCEEDED, should have failed
Negative Test http://www.w3.org/2000/10/rdf-tests/rdfcore/rdfms-empty-property-elements/Manifest.rdf#error002 SUCCEEDED, should have failed
Negative Test http://www.w3.org/2000/10/rdf-tests/rdfcore/rdfms-empty-property-elements/Manifest.rdf#error003 SUCCEEDED, should have failed
Negative Test http://www.w3.org/2000/10/rdf-tests/rdfcore/rdfms-syntax-incomplete/Manifest.rdf#error006 SUCCEEDED, should have failed

My bugs


Negative Test http://www.w3.org/2000/10/rdf-tests/rdfcore/xmlbase/Manifest.rdf#error001 SUCCEEDED, should have failed

The N-Triple Raptor returned was:
  <mailto:relfile> <http://www.w3.org/1999/02/22-rdf-syntax-ns#type> <http://example.org/type> .

[[
  Description: xml:base must be in generic syntax for resolution 
               of relative URI.
]] -- http://www.w3.org/2000/10/rdf-tests/rdfcore/xmlbase/error001.rdf

I can't see how to test this without knowledge of the specific URI
scheme here, 'mailto' and that it is not hierarchical.  The URI
should be changed to 'example:blah' unless there is something about
the mailto scheme that requires this.



Results for Negative Parser Tests
BAD       14   24.56%
OK        43   75.43%
TOTAL     57  100.00%

Received on Monday, 16 December 2002 09:08:52 UTC