pyRDFa Test Suite Reviewer

I have had a very hard time reading each RDFa test suite XHTML file,
SPARQL file and attempting to determine what triples should be generated
from the given XHTML. We might expect beginning RDFa implementers to
have the same problem. The test suite is not only good for validation,
but for understanding how RDFa works.

With the previous problems/thoughts in mind, I hacked together a quick
and dirty test suite reviewer. The reviewer is attached, along with two
examples of the output of the tool. It builds upon Ivan Herman's work on
pyRDFa and Ed Summer's modifications to the tool.

The new tool, tsreview.py, will do the following:
 - Retrieve all approved or unreviewed RDFa test cases
 - Execute the SPARQL queries on the generated graph
 - Dump the graph, in N3 format, to a file.
 - Dump the raw XHTML and raw SPARQL to a file.
 - Generate a test report, containing all the information necessary
   (XHTML, N3, and SPARQL), to understand whether the test case is
   behaving or not.

All of the current N3 and review files are included in the attached
.tar.bz2, in a directory called "review".

Two of the files that are generated, as a part of each test, are
important to implementers.

1. The review file very clearly shows the input, the output in N3, and
   the SPARQL validation code. This will help developers understand
   what is supposed to happen in their parsers.
2. The graph N3 serialization is provided. This ensures that developers
   won't have to write a SPARQL engine to test whether or not their
   parser is compliant.

Could we include the auto-generated N3 serialization as part of the RDFa
test suite (for documentation purposes)?

Could we include the review files (or something similar) as part of the
RDFa test suite (again, for documentation purposes)?

------------------------------------------------------------------
Command line output from the tsreview.py tool:

test-01-approved (__main__.XhtmlTests) ... ok
test-06-approved (__main__.XhtmlTests) ... ok
test-07-approved (__main__.XhtmlTests) ... ok
test-08-approved (__main__.XhtmlTests) ... ok
test-09-approved (__main__.XhtmlTests) ... ok
test-10-approved (__main__.XhtmlTests) ... ok
test-11-approved (__main__.XhtmlTests) ... FAIL
test-12-approved (__main__.XhtmlTests) ... FAIL
test-13-approved (__main__.XhtmlTests) ... FAIL
test-14-approved (__main__.XhtmlTests) ... FAIL
test-15-approved (__main__.XhtmlTests) ... ok
test-18-approved (__main__.XhtmlTests) ... ok
test-19-approved (__main__.XhtmlTests) ... ok
test-20-approved (__main__.XhtmlTests) ... ok
test-21-approved (__main__.XhtmlTests) ... ok
test-23-approved (__main__.XhtmlTests) ... ok
test-25-approved (__main__.XhtmlTests) ... ok
test-26-approved (__main__.XhtmlTests) ... ok
test-27-approved (__main__.XhtmlTests) ... ok
test-29-approved (__main__.XhtmlTests) ... FAIL
test-30-approved (__main__.XhtmlTests) ... ok
test-31-approved (__main__.XhtmlTests) ... ok
test-32-approved (__main__.XhtmlTests) ... ok
test-33-approved (__main__.XhtmlTests) ... FAIL
test-34-approved (__main__.XhtmlTests) ... ok
test-35-approved (__main__.XhtmlTests) ... ok
test-36-approved (__main__.XhtmlTests) ... ok
test-37-approved (__main__.XhtmlTests) ... ok
test-38-approved (__main__.XhtmlTests) ... ok
test-41-approved (__main__.XhtmlTests) ... ok
test-46-unreviewed (__main__.XhtmlTests) ... FAIL
test-47-unreviewed (__main__.XhtmlTests) ... ok
test-48-unreviewed (__main__.XhtmlTests) ... FAIL
test-49-unreviewed (__main__.XhtmlTests) ... ok
test-50-unreviewed (__main__.XhtmlTests) ... FAIL
test-51-unreviewed (__main__.XhtmlTests) ... FAIL
test-52-unreviewed (__main__.XhtmlTests) ... ok
test-53-unreviewed (__main__.XhtmlTests) ... ok
test-54-unreviewed (__main__.XhtmlTests) ... ok
test-55-unreviewed (__main__.XhtmlTests) ... ok
test-56-unreviewed (__main__.XhtmlTests) ... ERROR

-- manu

-- 
Manu Sporny
President/CEO - Digital Bazaar, Inc.
blog: Bitmunk Launches World's First Open Music Recommendation Service
http://blog.digitalbazaar.com/2007/09/09/bitmunk-music-recommendation/