Re: review of json-ld-syntax

On Mar 7, 2013, at 6:26 AM, Andy Seaborne <andy.seaborne@epimorphics.com> wrote:

> 
> 
> On 05/03/13 05:23, Sandro Hawke wrote:
>> > Conformance
>> 
>> SERIOUS
>> 
>> It's somewhat odd that all one needs for conformance is appendix B.
>> So what are the other normative parts of this document for...?
>> 
>> I think there may be a notion of a conformant JSON-LD generator or
>> parser here, too -- one that follows the rules of the rest of this
>> spec. That should be stated here.
> 
> Given the experience of the Turtle last call, getting a test suite for this document that is clearly signposted and delineated will save time later, together with instructions on running, then submitting results.
> 
> Having been working with tristan/jsonld-java recently [1], I looked to find the latest tests relevant to this document.  I can't determine what that is.

The test-suite really tests the algorithms in the JSON-LD processing document. We don't have syntax-only tests, but that might be reasonable to reference from the JSON-LD Syntax document; not a lot of time to put this together, though.

Also, I note that the HTML page doesn't directly reference the JSON-LD manifests. We could also import the JSON manifest definitions and use that to construct some HTML to provide human-level descriptions of each manifest and the associated tests. I do something like this for the EARL reports, so doing it for showing the test manifests should be easy enough.

> test-suite/manifest.jsonld is all tests.  I'm not sure of the status of the files and there

All of the tests referenced from these manifests are evaluation tests. The tests are typed, so the test intention is defined. For instance in <http://json-ld.org/test-suite/tests/expand-manifest.jsonld>, the tests are all defined as "jld:ExpandTest", indicating that these are evaluation tests for the expansion algorithms. I think the take-away is that the vocabulary terms and more details of how to perform testing should be referenced from the Test Suite HTML page.

> Will all the *jsonld files form syntax tests?

All evaluation tests can be considered as syntax tests. This doesn't provide for negative syntax tests, but because all JSON is, at some level, valid, these aren't as useful. Many evaluation tests include test to check that things that are not recognized (such as @foo) do not result in unexpected output. The API defines specific exceptions to be raised, but we have so far not figured out how to define test for this. In any case, from an algorithmic perspective, exceptions are not included (that's API only, not algorithm definition), so the best test IMO is to verify that the correct output is produced, which may be an empty document.

> But wouldn't that need freezing them as other work progresses?

Freezing these tese is really no different then freezing Turtle or JSON-LD tests. It could be that we want to create sub-directories, say for JSON-LD 1.0 tests, to allow a namespace for 1.1 tests in the future; AFAIK, Turtle doesn't do this, but SPARQL does, and it is useful.

> Will there be serialization tests (RDF->JSON-LD)?

Yes, RDF-JSON-LD tests are in <http://json-ld.org/test-suite/tests/fromRdf-manifest.jsonld>.

Gregg

> 	Andy
> 
> [1] https://github.com/afs/jena-jsonld#readme
> 
> PPS
> test-suite/index.html:
> 
> "rdf tests have input and sparql documents."
> ==>
> "rdf tests have input and NQuads output documents."

Thanks, a bit of old cruft.

Received on Thursday, 7 March 2013 16:10:57 UTC