Re: Using webservices for RDFa test suite validation

Wow, that was quick!:-)

Two small comments

- at least for the case of pyRdfa, I would think that the RDF/XML should
be the default return format. As I write on the page, rdflib's N3/Turtle
output is sometimes quite strange (look at Michael's mail). Or did I
misunderstand and is this already what you do?

- I presume it would be easy to change/update the code so that the URI
for the RDFa service could be given, eg, on the command line. Ie, the
same tool could be used for a bunch of other implementation, which could
generate output reports very easily for all of them...

Thanks a lot. I think this is really useful. I also copied this to Ed,
because he may not read this list directly, and he initiated this thread...

Ivan (maybe crazy, too...)

Manu Sporny wrote:
> Here's a proof-of-concept implementation of what Ivan had mentioned
> earlier today in the following thread:
> 
> http://www.nabble.com/pyRDFa-Test-Suite-Reviewer-tf4631832.html#a13226355
> 
> The new tool, crazyivan.py[1], will do the following:
>  - Retrieve all approved or unreviewed RDFa test cases from the W3C.
>  - Use the pyRDFa Distiller web service[2] to convert to RDF and run a
>    modified SPARQL query from the test suite using the sparqler
>    interface[3].
>  - Dump the XHTML, N3 triples, RDF, and SPARQL to separate files.
>  - Generate a test report, containing all the information necessary
>    (XHTML, N3, RDF, and SPARQL), to understand the test case.
> 
> Things to note about this implementation:
>  - It uses RDF and SPARQL web services to run the entire test suite.
>  - Many more of the tests succeed now that we're using up-to-date
>    implementations.
>  - Sometimes (roughly 1 in 500 tests) a unit test will fail because
>    (surprise, surprise) the HTTP GET fails due to some sort of
>    internal website/webservice error.
> 
> All of the current XHTML, N3, RDF and review files are included in the
> attached .tar.bz2, in a directory called "online-review".
> 
> The test doesn't require anything but a working version of Python and
> python-rdflib (and that's only because the test RDF manifest has to be
> queried).
> 
> [1] http://en.wikipedia.org/wiki/Crazy_Ivan
> [2] http://www.w3.org/2007/08/pyRdfa/
> [3] http://www.sparql.org/sparql.html
> 
> ------------------------------------------------------------------
> Command line output from the crazyivan.py tool:
> 
> test-online-01-approved (__main__.RDFaOnlineTest) ... ok
> test-online-06-approved (__main__.RDFaOnlineTest) ... ok
> test-online-07-approved (__main__.RDFaOnlineTest) ... ok
> test-online-08-approved (__main__.RDFaOnlineTest) ... ok
> test-online-09-approved (__main__.RDFaOnlineTest) ... ok
> test-online-10-approved (__main__.RDFaOnlineTest) ... ok
> test-online-11-approved (__main__.RDFaOnlineTest) ... FAIL
> test-online-12-approved (__main__.RDFaOnlineTest) ... ok
> test-online-13-approved (__main__.RDFaOnlineTest) ... ok
> test-online-14-approved (__main__.RDFaOnlineTest) ... ok
> test-online-15-approved (__main__.RDFaOnlineTest) ... ok
> test-online-18-approved (__main__.RDFaOnlineTest) ... ok
> test-online-19-approved (__main__.RDFaOnlineTest) ... ok
> test-online-20-approved (__main__.RDFaOnlineTest) ... ok
> test-online-21-approved (__main__.RDFaOnlineTest) ... ok
> test-online-23-approved (__main__.RDFaOnlineTest) ... ok
> test-online-25-approved (__main__.RDFaOnlineTest) ... ok
> test-online-26-approved (__main__.RDFaOnlineTest) ... ok
> test-online-27-approved (__main__.RDFaOnlineTest) ... ok
> test-online-29-approved (__main__.RDFaOnlineTest) ... FAIL
> test-online-30-approved (__main__.RDFaOnlineTest) ... ok
> test-online-31-approved (__main__.RDFaOnlineTest) ... ok
> test-online-32-approved (__main__.RDFaOnlineTest) ... ok
> test-online-33-approved (__main__.RDFaOnlineTest) ... ok
> test-online-34-approved (__main__.RDFaOnlineTest) ... ok
> test-online-35-approved (__main__.RDFaOnlineTest) ... ok
> test-online-36-approved (__main__.RDFaOnlineTest) ... ok
> test-online-37-approved (__main__.RDFaOnlineTest) ... ok
> test-online-38-approved (__main__.RDFaOnlineTest) ... ok
> test-online-41-approved (__main__.RDFaOnlineTest) ... ok
> test-online-46-unreviewed (__main__.RDFaOnlineTest) ... ok
> test-online-47-unreviewed (__main__.RDFaOnlineTest) ... ok
> test-online-48-unreviewed (__main__.RDFaOnlineTest) ... ok
> test-online-49-unreviewed (__main__.RDFaOnlineTest) ... ok
> test-online-50-unreviewed (__main__.RDFaOnlineTest) ... FAIL
> test-online-51-unreviewed (__main__.RDFaOnlineTest) ... ok
> test-online-52-unreviewed (__main__.RDFaOnlineTest) ... ok
> test-online-53-unreviewed (__main__.RDFaOnlineTest) ... ok
> test-online-54-unreviewed (__main__.RDFaOnlineTest) ... ok
> test-online-55-unreviewed (__main__.RDFaOnlineTest) ... ok
> test-online-56-unreviewed (__main__.RDFaOnlineTest) ... ERROR
> 
> -- manu
> 
> 
> 
> ------------------------------------------------------------------------
> 
> 
>   XHTML
> 
> <?xml version="1.0" encoding="UTF-8"?>
> <!DOCTYPE html PUBLIC "-//W3C//DTD XHTML+RDFa 1.0//EN" "http://www.w3.org/MarkUp/DTD/xhtml-rdfa-1.dtd">
> <html xmlns="http://www.w3.org/1999/xhtml"
>       xmlns:foaf="http://xmlns.com/foaf/0.1/">
>   <head>
>   <title>Test 0037</title>    
>   </head>
>   <body>
>    <div>
>       <img  about="http://sw-app.org/mic.xhtml#i"
>         rel="foaf:img"
>         src="http://sw-app.org/img/mic_2007_01.jpg" 
>         href="http://sw-app.org/img/mic_2006_03.jpg"
>         resource="http://sw-app.org/mic.xhtml#photo" 
>         alt="A photo depicting Michael" />   
>    </div>
>   </body>
> </html>
> 
>   
> 
> 
>   N3 Triples
> 
> 
> @prefix _5: <http://sw-app.org/mic.xhtml#>.
> @prefix foaf: <http://xmlns.com/foaf/0.1/>.
> 
>  _5:i foaf:img _5:photo. 
> 
>   
> 
> 
>   RDF
> 
> <?xml version="1.0" encoding="utf-8"?>
> <rdf:RDF
>   xmlns:foaf='http://xmlns.com/foaf/0.1/'
>   xmlns:rdf='http://www.w3.org/1999/02/22-rdf-syntax-ns#'
>>
>   <rdf:Description rdf:about="http://sw-app.org/mic.xhtml#i">
>     <foaf:img rdf:resource="http://sw-app.org/mic.xhtml#photo"/>
>   </rdf:Description>
> </rdf:RDF>
> 
>   
> 
> 
>   SPARQL (PASSED)
> 
> ASK FROM <http://www.w3.org/2007/08/pyRdfa/extract?uri=http%3A//www.w3.org/2006/07/SWD/RDFa/testsuite/xhtml1-testcases/0037.xhtml> WHERE {
>  <http://sw-app.org/mic.xhtml#i> <http://xmlns.com/foaf/0.1/img> <http://sw-app.org/mic.xhtml#photo> .
> }
>   
> 
> 
> ------------------------------------------------------------------------
> 
> 
>   XHTML
> 
> <?xml version="1.0" encoding="UTF-8"?>
> <!DOCTYPE html PUBLIC "-//W3C//DTD XHTML+RDFa 1.0//EN" "http://www.w3.org/MarkUp/DTD/xhtml-rdfa-1.dtd">
> <html xmlns="http://www.w3.org/1999/xhtml"
>       xmlns:dc="http://purl.org/dc/elements/1.1/">
>  <head>
>   <title>Test 0011</title>
>  </head>
>   <body>
>    <div about="">
>       Author: <span property="dc:creator">Albert Einstein</span>
>       <h2 property="dc:title">
>         E = mc<sup>2</sup>: The Most Urgent Problem of Our Time
>       </h2>
>  </div>
>   </body>
> </html>
> 
>   
> 
> 
>   N3 Triples
> 
> 
> @prefix _5: <http://www.w3.org/2006/07/SWD/RDFa/testsuite/xhtml1-testcases/0011.>.
> @prefix dc: <http://purl.org/dc/elements/1.1/>.
> 
>  _5:xhtml dc:creator "Albert Einstein";
>      dc:title """
>         E = mc<sup>2</sup>: The Most Urgent Problem of Our Time
>       """^^<http://www.w3.org/1999/02/22-rdf-syntax-ns#XMLLiteral>. 
> 
>   
> 
> 
>   RDF
> 
> <?xml version="1.0" encoding="utf-8"?>
> <rdf:RDF
>   xmlns:rdf='http://www.w3.org/1999/02/22-rdf-syntax-ns#'
>   xmlns:dc='http://purl.org/dc/elements/1.1/'
>>
>   <rdf:Description rdf:about="http://www.w3.org/2006/07/SWD/RDFa/testsuite/xhtml1-testcases/0011.xhtml">
>     <dc:creator>Albert Einstein</dc:creator>
>     <dc:title rdf:datatype="http://www.w3.org/1999/02/22-rdf-syntax-ns#XMLLiteral"><![CDATA[
>         E = mc<sup>2</sup>: The Most Urgent Problem of Our Time
>       ]]></dc:title>
>   </rdf:Description>
> </rdf:RDF>
> 
>   
> 
> 
>   SPARQL (FAILED)
> 
> ASK FROM <http://www.w3.org/2007/08/pyRdfa/extract?uri=http%3A//www.w3.org/2006/07/SWD/RDFa/testsuite/xhtml1-testcases/0011.xhtml> WHERE {
>  <http://www.w3.org/2006/07/SWD/RDFa/testsuite/xhtml1-testcases/0011.xhtml> <http://purl.org/dc/elements/1.1/creator> "Albert Einstein" .
>  <http://www.w3.org/2006/07/SWD/RDFa/testsuite/xhtml1-testcases/0011.xhtml> <http://purl.org/dc/elements/1.1/title> "E = mc<sup>2</sup>: The Most Urgent Problem of Our Time"^^<http://www.w3.org/1999/02/22-rdf-syntax-ns#XMLLiteral> .
> }
>   
> 

-- 

Ivan Herman, W3C Semantic Web Activity Lead
Home: http://www.w3.org/People/Ivan/
PGP Key: http://www.ivan-herman.net/pgpkey.html
FOAF: http://www.ivan-herman.net/foaf.rdf

Received on Wednesday, 17 October 2007 13:08:24 UTC