Re: pyRDFa Test Suite Reviewer

Hausenblas, Michael wrote:
> Ah. Sounds we have a volunteer, here ;)
> 

:-) I fly to Madrid tomorrow, US next week:-(

> So, to coordinate these efforts, may I propose
> to raise this issue at our upcoming RDFa TC
> (Manu will be there anyway, right? Ivan might
> want to chime in?)
> 

Not sure.

> I am personally also in favour of using an online
> service as sparql.org ...
> 
> However, a command-line version (IIUC that is what
> Manu offers) would be great to have ...
> 

All what I said can be incorporated into a command line script.
Actually, that _is_ probably the best way to go...

Ivan


> Cheers,
>  Michael
> 
> ----------------------------------------------------------
>  Michael Hausenblas, MSc.
>  Institute of Information Systems & Information Management
>  JOANNEUM RESEARCH Forschungsgesellschaft mbH
>   
>  http://www.joanneum.at/iis/
> ----------------------------------------------------------
>  
> 
>> -----Original Message-----
>> From: public-rdf-in-xhtml-tf-request@w3.org 
>> [mailto:public-rdf-in-xhtml-tf-request@w3.org] On Behalf Of Ivan Herman
>> Sent: Tuesday, October 16, 2007 9:37 AM
>> To: Manu Sporny
>> Cc: RDFa mailing list
>> Subject: Re: pyRDFa Test Suite Reviewer
>>
>> Manu,
>>
>> I was thinking of something similar, although my strategy was meant to
>> be a bit different
>>
>> - I would do the RDFa->RDF conversion using an online service. I guess
>> it would be possible to send a URI to either Fabien's service or pyRdfa
>> (well, I know about the latter) or others. In other words, a proper URI
>> for a test result could be minted. For example,
>>
>> http://www.w3.org/2007/08/pyRdfa/extract?uri=http%3A//www.w3.or
> g/2006/07/SWD/RDFa/testsuite/xhtml1-testcases/0012.xhtml
>> is a URI for test #12 with pyRdfa.
>>
>> - I would create a full SPARQL query using that URI in a 'FROM'
>>
>> - submit the SPARQL query to an online SPARQL site, like sparqler
>>
>> http://www.sparql.org/
>>
>> (I have created a SPARQL library to make such submission easier:
>>
>> http://ivanherman.wordpress.com/2007/07/06/sparql-endpoint-inte
> rface-to-python/
>> that could come handy here. But there are others.)
>>
>> This would have, in my view, several advantages:
>>
>> - the test would be done on the latest versions. At least for pyRdfa I
>> know for a fact that the version running on the web is ahead of the
>> tar.gz distribution
>>
>> - the same test could be done on different implementations 
>> easily with a
>> minor change in the target URI
>>
>> - the test would rely on an external and, possibly, more 
>> reliable sparql
>> service.
>>
>> As an example, you can send to sparqler the following query:
>>
>> ASK
>> FROM
>> <http://www.w3.org/2007/08/pyRdfa/extract?uri=http%3A//www.w3.o
> rg/2006/07/SWD/RDFa/testsuite/xhtml1-testcases/0012.xhtml>
>> WHERE {
>>  <http://example.org/node> <http://example.org/property> 
>> "chat"@fr .
>> }
>>
>> to test pyRdfa and test #12.
>>
>> Actually... I picked on #12 because it is also an example where the
>> python SPARQL goes wrong. Indeed, the sparqler result above is 'True'.
>> However, you test lists it as failing. For reference, here is 
>> the result
>> in Turtle generated by PyRDFa:
>>
>> [[[
>> @prefix ex: <http://example.org/>.
>>
>> ex:node ex:property "chat"@fr.
>> ]]]
>>
>> Which _is_ correct with the test above:-)
>>
>> We will have something nice here.
>>
>> Ivan
>>
>>
>> Manu Sporny wrote:
>>> I have had a very hard time reading each RDFa test suite XHTML file,
>>> SPARQL file and attempting to determine what triples should 
>> be generated
>>> from the given XHTML. We might expect beginning RDFa implementers to
>>> have the same problem. The test suite is not only good for 
>> validation,
>>> but for understanding how RDFa works.
>>>
>>> With the previous problems/thoughts in mind, I hacked 
>> together a quick
>>> and dirty test suite reviewer. The reviewer is attached, 
>> along with two
>>> examples of the output of the tool. It builds upon Ivan 
>> Herman's work on
>>> pyRDFa and Ed Summer's modifications to the tool.
>>>
>>> The new tool, tsreview.py, will do the following:
>>>  - Retrieve all approved or unreviewed RDFa test cases
>>>  - Execute the SPARQL queries on the generated graph
>>>  - Dump the graph, in N3 format, to a file.
>>>  - Dump the raw XHTML and raw SPARQL to a file.
>>>  - Generate a test report, containing all the information necessary
>>>    (XHTML, N3, and SPARQL), to understand whether the test case is
>>>    behaving or not.
>>>
>>> All of the current N3 and review files are included in the attached
>>> .tar.bz2, in a directory called "review".
>>>
>>> Two of the files that are generated, as a part of each test, are
>>> important to implementers.
>>>
>>> 1. The review file very clearly shows the input, the output 
>> in N3, and
>>>    the SPARQL validation code. This will help developers understand
>>>    what is supposed to happen in their parsers.
>>> 2. The graph N3 serialization is provided. This ensures that 
>> developers
>>>    won't have to write a SPARQL engine to test whether or not their
>>>    parser is compliant.
>>>
>>> Could we include the auto-generated N3 serialization as part 
>> of the RDFa
>>> test suite (for documentation purposes)?
>>>
>>> Could we include the review files (or something similar) as 
>> part of the
>>> RDFa test suite (again, for documentation purposes)?
>>>
>>> ------------------------------------------------------------------
>>> Command line output from the tsreview.py tool:
>>>
>>> test-01-approved (__main__.XhtmlTests) ... ok
>>> test-06-approved (__main__.XhtmlTests) ... ok
>>> test-07-approved (__main__.XhtmlTests) ... ok
>>> test-08-approved (__main__.XhtmlTests) ... ok
>>> test-09-approved (__main__.XhtmlTests) ... ok
>>> test-10-approved (__main__.XhtmlTests) ... ok
>>> test-11-approved (__main__.XhtmlTests) ... FAIL
>>> test-12-approved (__main__.XhtmlTests) ... FAIL
>>> test-13-approved (__main__.XhtmlTests) ... FAIL
>>> test-14-approved (__main__.XhtmlTests) ... FAIL
>>> test-15-approved (__main__.XhtmlTests) ... ok
>>> test-18-approved (__main__.XhtmlTests) ... ok
>>> test-19-approved (__main__.XhtmlTests) ... ok
>>> test-20-approved (__main__.XhtmlTests) ... ok
>>> test-21-approved (__main__.XhtmlTests) ... ok
>>> test-23-approved (__main__.XhtmlTests) ... ok
>>> test-25-approved (__main__.XhtmlTests) ... ok
>>> test-26-approved (__main__.XhtmlTests) ... ok
>>> test-27-approved (__main__.XhtmlTests) ... ok
>>> test-29-approved (__main__.XhtmlTests) ... FAIL
>>> test-30-approved (__main__.XhtmlTests) ... ok
>>> test-31-approved (__main__.XhtmlTests) ... ok
>>> test-32-approved (__main__.XhtmlTests) ... ok
>>> test-33-approved (__main__.XhtmlTests) ... FAIL
>>> test-34-approved (__main__.XhtmlTests) ... ok
>>> test-35-approved (__main__.XhtmlTests) ... ok
>>> test-36-approved (__main__.XhtmlTests) ... ok
>>> test-37-approved (__main__.XhtmlTests) ... ok
>>> test-38-approved (__main__.XhtmlTests) ... ok
>>> test-41-approved (__main__.XhtmlTests) ... ok
>>> test-46-unreviewed (__main__.XhtmlTests) ... FAIL
>>> test-47-unreviewed (__main__.XhtmlTests) ... ok
>>> test-48-unreviewed (__main__.XhtmlTests) ... FAIL
>>> test-49-unreviewed (__main__.XhtmlTests) ... ok
>>> test-50-unreviewed (__main__.XhtmlTests) ... FAIL
>>> test-51-unreviewed (__main__.XhtmlTests) ... FAIL
>>> test-52-unreviewed (__main__.XhtmlTests) ... ok
>>> test-53-unreviewed (__main__.XhtmlTests) ... ok
>>> test-54-unreviewed (__main__.XhtmlTests) ... ok
>>> test-55-unreviewed (__main__.XhtmlTests) ... ok
>>> test-56-unreviewed (__main__.XhtmlTests) ... ERROR
>>>
>>> -- manu
>>>
>>>
>>>
>>>
>> ---------------------------------------------------------------
>> ---------
>>>
>>>   XHTML
>>>
>>> <?xml version="1.0" encoding="UTF-8"?>
>>> <!DOCTYPE html PUBLIC "-//W3C//DTD XHTML+RDFa 1.0//EN" 
>> "http://www.w3.org/MarkUp/DTD/xhtml-rdfa-1.dtd">
>>> <html xmlns="http://www.w3.org/1999/xhtml"
>>>   xmlns:cc="http://creativecommons.org/ns#">
>>>  <head>
>>>   <title>Test 0008</title>
>>>  </head>
>>>   <body>
>>>    <p>This document is licensed under a
>>>     <a  about="" rel="cc:license"
>>>       
>> href="http://creativecommons.org/licenses/by-nc-nd/2.5/">
>>>          Creative Commons
>>>     </a>.
>>>    </p> 
>>>   </body>
>>> </html>
>>>
>>>   
>>>
>>>
>>>   Triples (N3 format)
>>>
>>>
>>> @prefix _5: 
>> <http://www.w3.org/2006/07/SWD/RDFa/testsuite/xhtml1-testcases/0008.>.
>>> @prefix cc: <http://creativecommons.org/ns#>.
>>>
>>>  _5:xhtml cc:license 
>> <http://creativecommons.org/licenses/by-nc-nd/2.5/>. 
>>>   
>>>
>>>
>>>   SPARQL (PASSED)
>>>
>>> ASK WHERE {
>>>  
>> <http://www.w3.org/2006/07/SWD/RDFa/testsuite/xhtml1-testcases/
> 0008.xhtml> <http://creativecommons.org/ns#license>
>> <http://creativecommons.org/licenses/by-nc-nd/2.5/> .
>>> }
>>>   
>>>
>>>
>>>
>> ---------------------------------------------------------------
>> ---------
>>>
>>>   XHTML
>>>
>>> <?xml version="1.0" encoding="UTF-8"?>
>>> <!DOCTYPE html PUBLIC "-//W3C//DTD XHTML+RDFa 1.0//EN" 
>> "http://www.w3.org/MarkUp/DTD/xhtml-rdfa-1.dtd">
>>> <html xmlns="http://www.w3.org/1999/xhtml"
>>>       xmlns:foaf="http://xmlns.com/foaf/0.1/"
>>>    >
>>>   <head>
>>>   <title>Test 0048</title>    
>>>   </head>
>>>   <body>
>>>    <div about="http://www.example.org" rel="foaf:knows" 
>> instanceof="foaf:Person">
>>>      <p property="foaf:name">John Doe</p>
>>>     </div>
>>>   </body>
>>> </html>
>>>
>>>   
>>>
>>>
>>>   Triples (N3 format)
>>>
>>>
>>> @prefix foaf: <http://xmlns.com/foaf/0.1/>.
>>> @prefix rdf: <http://www.w3.org/1999/02/22-rdf-syntax-ns#>.
>>>
>>>  <http://www.example.org> a foaf:Person;
>>>      foaf:knows [ foaf:name "John Doe"]. 
>>>   
>>>
>>>
>>>   SPARQL (FAILED)
>>>
>>> ASK WHERE {
>>>  <http://www.example.org> <http://xmlns.com/foaf/0.1/knows> _:a .
>>>  _:a <http://xmlns.com/foaf/0.1/name> "John Doe" .
>>>  <http://www.example.org> 
>> <http://www.w3.org/1999/02/22-rdf-syntax-ns#type> 
>> <http://xmlns.com/foaf/0.1/Person> .
>>> }
>>>   
>>>
>> -- 
>>
>> Ivan Herman, W3C Semantic Web Activity Lead
>> Home: http://www.w3.org/People/Ivan/
>> PGP Key: http://www.ivan-herman.net/pgpkey.html
>> FOAF: http://www.ivan-herman.net/foaf.rdf
>>

-- 

Ivan Herman, W3C Semantic Web Activity Lead
Home: http://www.w3.org/People/Ivan/
PGP Key: http://www.ivan-herman.net/pgpkey.html
FOAF: http://www.ivan-herman.net/foaf.rdf

Received on Tuesday, 16 October 2007 07:47:45 UTC