Re: Rollup EARL report for Turtle

Some general notes on the EARL report consolidator, and advice for submitting EARL reports:

The project README [1] includes some more information on expected structure of the test manifest and the test reports. To make things easier, please consider the following when creating the EARL report for your software:

* Include an earl:Assertion for each test run.
* The earl:test property of that assertion should reference a test description from the manifest, when loaded from it's origin <http://dvcs.w3.org/hg/rdf/raw-file/default/rdf-turtle/tests-ttl/manifest.ttl>. For example, the earl:test object for turtle-syntax-file-01 would be <http://dvcs.w3.org/hg/rdf/raw-file/default/rdf-turtle/tests-ttl/manifest.ttl#turtle-syntax-file-01>. (This could be made more independent by include a different @base in the manifest file).
* The earl:software property of the assertion should point to DOAP description of your project. If not included along with the EARL report, the software will attempt to load it from that location.
* The DOAP description should have a doap:developer containing a foaf:name used for reporting on the developer of the software. If the DOAP description includes a doap:homepage, this will be used to link to an appropriate location in the output report.

I'm open to suggestions on how to make the process simpler and more flexible. For example, it currently expects to use the Manifest structure we're using for Turtle, TriG, N-Triples and N-Quads. It could also work for SPARQL EARL reports, but would need a slightly different query to construct the EARL TestCases, and the RDFa output of the report would need a bit more flexibility. For example, the RDFa test manifest is substantially different, and the reporting software for that has corresponding changes. It could be that a modular design, where the important queries are loaded from a configuration file. The output template can already be specified at run-time.

Gregg Kellogg
gregg@greggkellogg.net<mailto:gregg@greggkellogg.net>

On Nov 17, 2012, at 5:37 PM, Gregg Kellogg <gregg@greggkellogg.net<mailto:gregg@greggkellogg.net>> wrote:

I've created software [1] to generate the rollup EARL report and generated such a report using my own Turtle implementation.

Generated report: http://dvcs.w3.org/hg/rdf/raw-file/default/rdf-turtle/reports/index.html
Ruby Turtle EARL report:  http://dvcs.w3.org/hg/rdf/raw-file/default/rdf-turtle/reports/rdf.rb-earl.ttl

There is information in the README [2] on how to run the generator with new reports. When completed, EARL reports can get checked into the mercurial repository, and the report can be re-generated.

More information on the expectation of the EARL reports, DOAP and FOAF records used to generate the report is in the earl-report GitHub repository [1].

Gregg Kellogg
gregg@greggkellogg.net<mailto:gregg@greggkellogg.net>

[1] https://github.com/gkellogg/earl-report
[2] http://dvcs.w3.org/hg/rdf/raw-file/default/rdf-turtle/reports/README