Re: Some tests for Turtle

FWIW, I'm now processing Andy's Turtle tests. I'm actually using JSON-LD to get an easily consumable version of the tests. The process is the following (using the Ruby RDF libraries):

Parse each manifest to an RDF Repository:

        g = RDF::Graph.load(file, :format => :turtle)

Generate JSON-LD from it and frame it. The frame I'm using is the following:

    {
      "@context": {
        "xsd": "http://www.w3.org/2001/XMLSchema#",
        "rdfs": "http://www.w3.org/2000/01/rdf-schema#",
        "mf": "http://www.w3.org/2001/sw/DataAccess/tests/test-manifest#",
        "mq": "http://www.w3.org/2001/sw/DataAccess/tests/test-query#",
        "rdft": "http://www.w3.org/ns/rdftest#",
    
        "comment": "rdfs:comment",
        "entries": {"@id": "mf:entries", "@container": "@list"},
        "name": "mf:name",
        "action": {"@id": "mf:action", "@type": "@id"},
        "result": {"@id": "mf:result", "@type": "@id"}
      },
      "@type": "mf:Manifest",
      "entries": {
        "@type": [
          "rdft:TestTurtlePositiveSyntax",
          "rdft:TestTurtleNegativeSyntax",
          "rdft:TestTurtlePositiveEval"
        ]
      }
    }

This basically puts all the tests within the "entries" property.

Then I generate the framed tests as follows:

        JSON::LD::API.fromRDF(g) do |expanded|
          JSON::LD::API.frame(expanded, FRAME) do |framed|
            yield Manifest.new(framed['@graph'].first)
          end
        end

Basically, turn the RDF into JSON-LD, apply the frame, and then yield (or callback) with the result. A sample of the converted manifest is the following:

{
  "@context": {
    "xsd": "http://www.w3.org/2001/XMLSchema#",
    "rdfs": "http://www.w3.org/2000/01/rdf-schema#",
    "mf": "http://www.w3.org/2001/sw/DataAccess/tests/test-manifest#",
    "mq": "http://www.w3.org/2001/sw/DataAccess/tests/test-query#",
    "rdft": "http://www.w3.org/ns/rdftest#",
    "comment": "rdfs:comment",
    "entries": {
      "@id": "mf:entries",
      "@container": "@list"
    },
    "name": "mf:name",
    "action": {
      "@id": "mf:action",
      "@type": "@id"
    },
    "result": {
      "@id": "mf:result",
      "@type": "@id"
    }
  },
  "@graph": [
    {
      "@id": "http://svn.apache.org/repos/asf/jena/Experimental/riot-reader/testing/RIOT/Lang/Turtle/manifest.ttl",
      "@type": "mf:Manifest",
      "comment": "Turtle tests",
      "entries": [
        {
          "@id": "http://svn.apache.org/repos/asf/jena/Experimental/riot-reader/testing/RIOT/Lang/Turtle/manifest.ttl#syn-file-01",
          "@type": "rdft:TestTurtlePositiveSyntax",
          "action": "http://svn.apache.org/repos/asf/jena/Experimental/riot-reader/testing/RIOT/Lang/Turtle/syn-file-01.ttl",
          "name": "syn-file-01"
        },
      ...
    }
  ]
}

I can then iterate through each entry in entries using the 'action', 'result', and '@type' to perform each test.

My processor still passes all of the old tests, but fails a number of the syntax tests, which is expected and now makes it easier for my to update my processor.

For anyone interested, my test runner is at https://github.com/ruby-rdf/rdf-turtle/blob/master/spec/suite_spec.rb. The bits to convert the manifests into JSON-LD (and a Ruby Resource class to access this) is at https://github.com/ruby-rdf/rdf-turtle/blob/master/spec/suite_helper.rb.

Thanks Andy!

Gregg Kellogg
gregg@greggkellogg.net

On Oct 12, 2012, at 9:05 AM, Andy Seaborne <andy.seaborne@epimorphics.com> wrote:

> Here are some tests for Turtle as a seed of a test suite.
> 
> https://svn.apache.org/repos/asf/jena/Experimental/riot-reader/testing/RIOT/Lang/
> 
> --> TurtleSubm
> 
> These are the tests from the Turtle submission, cleaned up.
> 
> --> Turtle
> 
> New syntax tests.  Work-in-progress.
> 
> - - - - - - - - -
> 
> The "jena/Experimental" area is not in the current release codebase
> and can changed at any time (i.e. no release code freeze)
> 
> == TurtleSubm/
> 
> These are the tests from the Turtle submission, cleaned up to make them 
> passable.  e.g. test 29 had a bad URI characters - parsers that do basic 
> character range checks as in the Turtle LC grammar would flag this as an 
> error.
> 
> TurtleSubm/manifest.ttl
> TurtleSubm/manifest-bad.ttl
> 
> == Turtle/
> 
> This is work-in-progress and I'll be adding more tests over the next few 
> days as time permits.
> 
> Turtle/manifest.ttl
> Current License: ASL2 (changing to W3C Software license is no problem)
> 
> Only syntax tests (positive and negative) and all tests are in a single 
> manifest file (good and bad syntax tests).
> 
> == Manifest
> 
> The tests use the manifest format which is a general framework or 
> "action" and "result".  Tests are typed. It was specialised and used by 
> SPARQL 1.0 and I know some other people use it for their tests other 
> than SPARQL.
> 
> But also the tests are systematically named (by and large) so there is 
> no need to write a complex test environment.  I hope it will be relative 
> simple to incorporate the tests into any environment with as little 
> overhead as possible.
> 
> The test suite serves two purposes : getting the Turtle spec through W3C 
> process but also as a community resource beyond the working group for 
> validating parsers.  Creating good coverage is some thing we can share.
> 
> 	Andy
>

Received on Monday, 22 October 2012 00:53:43 UTC