Re: student project idea: RDF/RDFa parser QA via automatic test-suite generation

Dan Brickley wrote:
> I'd like to see an auto-generated repository of RDFa samples, most (but
> not all) of which are decent wellformed XHTML with RDFa, but also with a
> good number of poorly-marked up files. 

+1 - sounds like a worthwhile project.

There are two permutations of this approach:

The first involves generating valid and invalid XHTML+RDFa to see if the
parsers can make it through the file. Did the parser dump core or did it
exit with a good status code?

The second involves generating valid XHTML+RDFa as well as the
corresponding SPARQL files such that they can be hooked up to the RDFa
Test harness. Did the parser exit with a good status code AND did the
SPARQL evaluate to TRUE?

> Generating such a test set and then wiring it up to a set of RDFa
> parsers (via http://rdfa.digitalbazaar.com/rdfa-test-harness/ or
> something like it) shouldn't be a huge job

It would be fairly straight-forward to do this - the RDFa Test Harness
is already setup for use-cases like what you are describing. We would need:

A manifest file[1], and a set of matching RDFa+XHTML files and their
corresponding SPARQL files[2].

> (c) whether the spec gurus agree on what ought to be generated.

I don't suggest getting the spec gurus involved in most of the 1000 test
cases. On the RDFa telecons, it takes us roughly 5-10 minutes to get
through the simple, straight-forward test cases... and that's after
we've reviewed them offline. I'd lean on the spec gurus only when there
is a disagreement between the parser writers on what should happen.

This would be a great summer project for a student. I'd be willing to
lend advice and help integrating with the RDFa Test Harness.

-- manu

[1]http://www.w3.org/2006/07/SWD/RDFa/testsuite/xhtml1-testcases/rdfa-xhtml1-test-manifest.rdf
[2]http://www.w3.org/2006/07/SWD/RDFa/testsuite/xhtml1-testcases/Test0001

-- 
Manu Sporny
President/CEO - Digital Bazaar, Inc.

blog: POSIX Threads Don't Scale Past 100K Concurrent Web Requests
http://blog.digitalbazaar.com/2008/09/30/scaling-webservices-part-1

blog: Fibers are the Future: Scaling Past 100K Concurrent Web Requests
http://blog.digitalbazaar.com/2008/10/21/scaling-webservices-part-2

Received on Tuesday, 18 November 2008 15:12:40 UTC