RE: pyRDFa Test Suite Reviewer

Manu,

Thanks a lot for this initiative. I have to admit that
we need to do here a *bit* more - your proposal is most welcome.

The current status is as follows:

+ We have the XHTML 1.1 TC Manifest document [1] as the only 
  authoritative source defining what the input (XHTML+RDFa)
  and what the output (SPARQL ASK yes/no) is.

+ The RDFa TC Wiki page [2] is used TF-internally for reviewing
  and to document the resolutions (note that this page is maintained
  manually)

+ The rdfa.debug tool [3] allows to view the input and output of a TC
  side by side (ok its not very sophisticated but helps a lot during the
  actual approval)

+ An RDFa Test Runner interface [4] aiming at basically the same
  what you propose (the actual testing is done on 'my' site implemented
  in PHP)

And now we have your proposal allowing to do an automated check and
offering a nice overview ...

I'm absolutely willing to support this (given someone helps me with the
Python stuff, I'm not familiar with). I already proposed to install a 
Test Runner at W3C - maybe this is our chance, now ;)

Please note that there is a pending action of mine to actually define 
what the semantics of a certain TC output is [5]. In my understanding 
we'll have to set up a list in the RDFa Syntax Document (Sec. 4.3.) [6]
to state (for each TC) if a SPARQL ASK 'yes' means 'failed' or 'passed'.

Cheers,
	Michael

[1]
http://www.w3.org/2006/07/SWD/RDFa/testsuite/xhtml1-testcases/rdfa-xhtml
1-test-manifest.rdf
[2] http://www.w3.org/2006/07/SWD/wiki/RDFaTC
[3] http://sw.joanneum.at/rdfa/rdfadebug/index.php
[4] http://www.w3.org/2006/07/SWD/RDFa/testsuite/run-tc-new.html
[5] http://www.w3.org/2007/09/06-rdfa-minutes.html#action07
[6] http://www.w3.org/MarkUp/2007/ED-rdfa-syntax-20070927/#processorconf

----------------------------------------------------------
 Michael Hausenblas, MSc.
 Institute of Information Systems & Information Management
 JOANNEUM RESEARCH Forschungsgesellschaft mbH
  
 http://www.joanneum.at/iis/
----------------------------------------------------------
 

>-----Original Message-----
>From: public-rdf-in-xhtml-tf-request@w3.org 
>[mailto:public-rdf-in-xhtml-tf-request@w3.org] On Behalf Of Manu Sporny
>Sent: Tuesday, October 16, 2007 5:44 AM
>To: RDFa mailing list
>Subject: pyRDFa Test Suite Reviewer
>
>I have had a very hard time reading each RDFa test suite XHTML file,
>SPARQL file and attempting to determine what triples should be 
>generated
>from the given XHTML. We might expect beginning RDFa implementers to
>have the same problem. The test suite is not only good for validation,
>but for understanding how RDFa works.
>
>With the previous problems/thoughts in mind, I hacked together a quick
>and dirty test suite reviewer. The reviewer is attached, along with two
>examples of the output of the tool. It builds upon Ivan 
>Herman's work on
>pyRDFa and Ed Summer's modifications to the tool.
>
>The new tool, tsreview.py, will do the following:
> - Retrieve all approved or unreviewed RDFa test cases
> - Execute the SPARQL queries on the generated graph
> - Dump the graph, in N3 format, to a file.
> - Dump the raw XHTML and raw SPARQL to a file.
> - Generate a test report, containing all the information necessary
>   (XHTML, N3, and SPARQL), to understand whether the test case is
>   behaving or not.
>
>All of the current N3 and review files are included in the attached
>.tar.bz2, in a directory called "review".
>
>Two of the files that are generated, as a part of each test, are
>important to implementers.
>
>1. The review file very clearly shows the input, the output in N3, and
>   the SPARQL validation code. This will help developers understand
>   what is supposed to happen in their parsers.
>2. The graph N3 serialization is provided. This ensures that developers
>   won't have to write a SPARQL engine to test whether or not their
>   parser is compliant.
>
>Could we include the auto-generated N3 serialization as part 
>of the RDFa
>test suite (for documentation purposes)?
>
>Could we include the review files (or something similar) as part of the
>RDFa test suite (again, for documentation purposes)?
>
>------------------------------------------------------------------
>Command line output from the tsreview.py tool:
>
>test-01-approved (__main__.XhtmlTests) ... ok
>test-06-approved (__main__.XhtmlTests) ... ok
>test-07-approved (__main__.XhtmlTests) ... ok
>test-08-approved (__main__.XhtmlTests) ... ok
>test-09-approved (__main__.XhtmlTests) ... ok
>test-10-approved (__main__.XhtmlTests) ... ok
>test-11-approved (__main__.XhtmlTests) ... FAIL
>test-12-approved (__main__.XhtmlTests) ... FAIL
>test-13-approved (__main__.XhtmlTests) ... FAIL
>test-14-approved (__main__.XhtmlTests) ... FAIL
>test-15-approved (__main__.XhtmlTests) ... ok
>test-18-approved (__main__.XhtmlTests) ... ok
>test-19-approved (__main__.XhtmlTests) ... ok
>test-20-approved (__main__.XhtmlTests) ... ok
>test-21-approved (__main__.XhtmlTests) ... ok
>test-23-approved (__main__.XhtmlTests) ... ok
>test-25-approved (__main__.XhtmlTests) ... ok
>test-26-approved (__main__.XhtmlTests) ... ok
>test-27-approved (__main__.XhtmlTests) ... ok
>test-29-approved (__main__.XhtmlTests) ... FAIL
>test-30-approved (__main__.XhtmlTests) ... ok
>test-31-approved (__main__.XhtmlTests) ... ok
>test-32-approved (__main__.XhtmlTests) ... ok
>test-33-approved (__main__.XhtmlTests) ... FAIL
>test-34-approved (__main__.XhtmlTests) ... ok
>test-35-approved (__main__.XhtmlTests) ... ok
>test-36-approved (__main__.XhtmlTests) ... ok
>test-37-approved (__main__.XhtmlTests) ... ok
>test-38-approved (__main__.XhtmlTests) ... ok
>test-41-approved (__main__.XhtmlTests) ... ok
>test-46-unreviewed (__main__.XhtmlTests) ... FAIL
>test-47-unreviewed (__main__.XhtmlTests) ... ok
>test-48-unreviewed (__main__.XhtmlTests) ... FAIL
>test-49-unreviewed (__main__.XhtmlTests) ... ok
>test-50-unreviewed (__main__.XhtmlTests) ... FAIL
>test-51-unreviewed (__main__.XhtmlTests) ... FAIL
>test-52-unreviewed (__main__.XhtmlTests) ... ok
>test-53-unreviewed (__main__.XhtmlTests) ... ok
>test-54-unreviewed (__main__.XhtmlTests) ... ok
>test-55-unreviewed (__main__.XhtmlTests) ... ok
>test-56-unreviewed (__main__.XhtmlTests) ... ERROR
>
>-- manu
>
>-- 
>Manu Sporny
>President/CEO - Digital Bazaar, Inc.
>blog: Bitmunk Launches World's First Open Music Recommendation Service
>http://blog.digitalbazaar.com/2007/09/09/bitmunk-music-recommendation/
>

Received on Tuesday, 16 October 2007 07:26:27 UTC