RE: pyRDFa Test Suite Reviewer

Ah. Sounds we have a volunteer, here ;)

So, to coordinate these efforts, may I propose
to raise this issue at our upcoming RDFa TC
(Manu will be there anyway, right? Ivan might
want to chime in?)

I am personally also in favour of using an online
service as sparql.org ...

However, a command-line version (IIUC that is what
Manu offers) would be great to have ...

Cheers,
	Michael

----------------------------------------------------------
 Michael Hausenblas, MSc.
 Institute of Information Systems & Information Management
 JOANNEUM RESEARCH Forschungsgesellschaft mbH
  
 http://www.joanneum.at/iis/
----------------------------------------------------------
 

>-----Original Message-----
>From: public-rdf-in-xhtml-tf-request@w3.org 
>[mailto:public-rdf-in-xhtml-tf-request@w3.org] On Behalf Of Ivan Herman
>Sent: Tuesday, October 16, 2007 9:37 AM
>To: Manu Sporny
>Cc: RDFa mailing list
>Subject: Re: pyRDFa Test Suite Reviewer
>
>Manu,
>
>I was thinking of something similar, although my strategy was meant to
>be a bit different
>
>- I would do the RDFa->RDF conversion using an online service. I guess
>it would be possible to send a URI to either Fabien's service or pyRdfa
>(well, I know about the latter) or others. In other words, a proper URI
>for a test result could be minted. For example,
>
>http://www.w3.org/2007/08/pyRdfa/extract?uri=http%3A//www.w3.or
g/2006/07/SWD/RDFa/testsuite/xhtml1-testcases/0012.xhtml
>
>is a URI for test #12 with pyRdfa.
>
>- I would create a full SPARQL query using that URI in a 'FROM'
>
>- submit the SPARQL query to an online SPARQL site, like sparqler
>
>http://www.sparql.org/
>
>(I have created a SPARQL library to make such submission easier:
>
>http://ivanherman.wordpress.com/2007/07/06/sparql-endpoint-inte
rface-to-python/
>
>that could come handy here. But there are others.)
>
>This would have, in my view, several advantages:
>
>- the test would be done on the latest versions. At least for pyRdfa I
>know for a fact that the version running on the web is ahead of the
>tar.gz distribution
>
>- the same test could be done on different implementations 
>easily with a
>minor change in the target URI
>
>- the test would rely on an external and, possibly, more 
>reliable sparql
>service.
>
>As an example, you can send to sparqler the following query:
>
>ASK
>FROM
><http://www.w3.org/2007/08/pyRdfa/extract?uri=http%3A//www.w3.o
rg/2006/07/SWD/RDFa/testsuite/xhtml1-testcases/0012.xhtml>
>WHERE {
>	<http://example.org/node> <http://example.org/property> 
>"chat"@fr .
>}
>
>to test pyRdfa and test #12.
>
>Actually... I picked on #12 because it is also an example where the
>python SPARQL goes wrong. Indeed, the sparqler result above is 'True'.
>However, you test lists it as failing. For reference, here is 
>the result
>in Turtle generated by PyRDFa:
>
>[[[
>@prefix ex: <http://example.org/>.
>
> ex:node ex:property "chat"@fr.
>]]]
>
>Which _is_ correct with the test above:-)
>
>We will have something nice here.
>
>Ivan
>
>
>Manu Sporny wrote:
>> I have had a very hard time reading each RDFa test suite XHTML file,
>> SPARQL file and attempting to determine what triples should 
>be generated
>> from the given XHTML. We might expect beginning RDFa implementers to
>> have the same problem. The test suite is not only good for 
>validation,
>> but for understanding how RDFa works.
>> 
>> With the previous problems/thoughts in mind, I hacked 
>together a quick
>> and dirty test suite reviewer. The reviewer is attached, 
>along with two
>> examples of the output of the tool. It builds upon Ivan 
>Herman's work on
>> pyRDFa and Ed Summer's modifications to the tool.
>> 
>> The new tool, tsreview.py, will do the following:
>>  - Retrieve all approved or unreviewed RDFa test cases
>>  - Execute the SPARQL queries on the generated graph
>>  - Dump the graph, in N3 format, to a file.
>>  - Dump the raw XHTML and raw SPARQL to a file.
>>  - Generate a test report, containing all the information necessary
>>    (XHTML, N3, and SPARQL), to understand whether the test case is
>>    behaving or not.
>> 
>> All of the current N3 and review files are included in the attached
>> .tar.bz2, in a directory called "review".
>> 
>> Two of the files that are generated, as a part of each test, are
>> important to implementers.
>> 
>> 1. The review file very clearly shows the input, the output 
>in N3, and
>>    the SPARQL validation code. This will help developers understand
>>    what is supposed to happen in their parsers.
>> 2. The graph N3 serialization is provided. This ensures that 
>developers
>>    won't have to write a SPARQL engine to test whether or not their
>>    parser is compliant.
>> 
>> Could we include the auto-generated N3 serialization as part 
>of the RDFa
>> test suite (for documentation purposes)?
>> 
>> Could we include the review files (or something similar) as 
>part of the
>> RDFa test suite (again, for documentation purposes)?
>> 
>> ------------------------------------------------------------------
>> Command line output from the tsreview.py tool:
>> 
>> test-01-approved (__main__.XhtmlTests) ... ok
>> test-06-approved (__main__.XhtmlTests) ... ok
>> test-07-approved (__main__.XhtmlTests) ... ok
>> test-08-approved (__main__.XhtmlTests) ... ok
>> test-09-approved (__main__.XhtmlTests) ... ok
>> test-10-approved (__main__.XhtmlTests) ... ok
>> test-11-approved (__main__.XhtmlTests) ... FAIL
>> test-12-approved (__main__.XhtmlTests) ... FAIL
>> test-13-approved (__main__.XhtmlTests) ... FAIL
>> test-14-approved (__main__.XhtmlTests) ... FAIL
>> test-15-approved (__main__.XhtmlTests) ... ok
>> test-18-approved (__main__.XhtmlTests) ... ok
>> test-19-approved (__main__.XhtmlTests) ... ok
>> test-20-approved (__main__.XhtmlTests) ... ok
>> test-21-approved (__main__.XhtmlTests) ... ok
>> test-23-approved (__main__.XhtmlTests) ... ok
>> test-25-approved (__main__.XhtmlTests) ... ok
>> test-26-approved (__main__.XhtmlTests) ... ok
>> test-27-approved (__main__.XhtmlTests) ... ok
>> test-29-approved (__main__.XhtmlTests) ... FAIL
>> test-30-approved (__main__.XhtmlTests) ... ok
>> test-31-approved (__main__.XhtmlTests) ... ok
>> test-32-approved (__main__.XhtmlTests) ... ok
>> test-33-approved (__main__.XhtmlTests) ... FAIL
>> test-34-approved (__main__.XhtmlTests) ... ok
>> test-35-approved (__main__.XhtmlTests) ... ok
>> test-36-approved (__main__.XhtmlTests) ... ok
>> test-37-approved (__main__.XhtmlTests) ... ok
>> test-38-approved (__main__.XhtmlTests) ... ok
>> test-41-approved (__main__.XhtmlTests) ... ok
>> test-46-unreviewed (__main__.XhtmlTests) ... FAIL
>> test-47-unreviewed (__main__.XhtmlTests) ... ok
>> test-48-unreviewed (__main__.XhtmlTests) ... FAIL
>> test-49-unreviewed (__main__.XhtmlTests) ... ok
>> test-50-unreviewed (__main__.XhtmlTests) ... FAIL
>> test-51-unreviewed (__main__.XhtmlTests) ... FAIL
>> test-52-unreviewed (__main__.XhtmlTests) ... ok
>> test-53-unreviewed (__main__.XhtmlTests) ... ok
>> test-54-unreviewed (__main__.XhtmlTests) ... ok
>> test-55-unreviewed (__main__.XhtmlTests) ... ok
>> test-56-unreviewed (__main__.XhtmlTests) ... ERROR
>> 
>> -- manu
>> 
>> 
>> 
>> 
>---------------------------------------------------------------
>---------
>> 
>> 
>>   XHTML
>> 
>> <?xml version="1.0" encoding="UTF-8"?>
>> <!DOCTYPE html PUBLIC "-//W3C//DTD XHTML+RDFa 1.0//EN" 
>"http://www.w3.org/MarkUp/DTD/xhtml-rdfa-1.dtd">
>> <html xmlns="http://www.w3.org/1999/xhtml"
>>   xmlns:cc="http://creativecommons.org/ns#">
>> 	<head>
>> 		<title>Test 0008</title>
>> 	</head>
>>   <body>
>>   	<p>This document is licensed under a
>>   		<a 	about="" rel="cc:license"
>>     		
>href="http://creativecommons.org/licenses/by-nc-nd/2.5/">
>>        		Creative Commons
>>   		</a>.
>>   	</p>	
>>   </body>
>> </html>
>> 
>>   
>> 
>> 
>>   Triples (N3 format)
>> 
>> 
>> @prefix _5: 
><http://www.w3.org/2006/07/SWD/RDFa/testsuite/xhtml1-testcases/0008.>.
>> @prefix cc: <http://creativecommons.org/ns#>.
>> 
>>  _5:xhtml cc:license 
><http://creativecommons.org/licenses/by-nc-nd/2.5/>. 
>>   
>> 
>> 
>>   SPARQL (PASSED)
>> 
>> ASK WHERE {
>> 	
><http://www.w3.org/2006/07/SWD/RDFa/testsuite/xhtml1-testcases/
0008.xhtml> <http://creativecommons.org/ns#license>
><http://creativecommons.org/licenses/by-nc-nd/2.5/> .
>> }
>>   
>> 
>> 
>> 
>---------------------------------------------------------------
>---------
>> 
>> 
>>   XHTML
>> 
>> <?xml version="1.0" encoding="UTF-8"?>
>> <!DOCTYPE html PUBLIC "-//W3C//DTD XHTML+RDFa 1.0//EN" 
>"http://www.w3.org/MarkUp/DTD/xhtml-rdfa-1.dtd">
>> <html xmlns="http://www.w3.org/1999/xhtml"
>>       xmlns:foaf="http://xmlns.com/foaf/0.1/"
>> 	  >
>>   <head>
>> 		<title>Test 0048</title>    
>>   </head>
>>   <body>
>>   	<div about="http://www.example.org" rel="foaf:knows" 
>instanceof="foaf:Person">
>> 	    <p property="foaf:name">John Doe</p>
>>    	</div>
>>   </body>
>> </html>
>> 
>>   
>> 
>> 
>>   Triples (N3 format)
>> 
>> 
>> @prefix foaf: <http://xmlns.com/foaf/0.1/>.
>> @prefix rdf: <http://www.w3.org/1999/02/22-rdf-syntax-ns#>.
>> 
>>  <http://www.example.org> a foaf:Person;
>>      foaf:knows [ foaf:name "John Doe"]. 
>>   
>> 
>> 
>>   SPARQL (FAILED)
>> 
>> ASK WHERE {
>> 	<http://www.example.org> <http://xmlns.com/foaf/0.1/knows> _:a .
>> 	_:a <http://xmlns.com/foaf/0.1/name> "John Doe" .
>> 	<http://www.example.org> 
><http://www.w3.org/1999/02/22-rdf-syntax-ns#type> 
><http://xmlns.com/foaf/0.1/Person> .
>> }
>>   
>> 
>
>-- 
>
>Ivan Herman, W3C Semantic Web Activity Lead
>Home: http://www.w3.org/People/Ivan/
>PGP Key: http://www.ivan-herman.net/pgpkey.html
>FOAF: http://www.ivan-herman.net/foaf.rdf
>

Received on Tuesday, 16 October 2007 07:44:51 UTC