RE: Testing API options

On Friday, August 23, 2013 1:32 AM, Gregg Kellogg wrote:
> We could create an "option" property, which takes a set of objects to
> define the option name and value to be passed in. Values are typically
> strings, but it may be useful to pass in arrays or hashes as values
> too; however, this would loose an RDF interpretation of the value, so
> we might not want to go there. This might look like the following:
> 
[...]
>           "@id": "#txxx",
>           "@type": ["jld:PositiveEvaluationTest", "jld:CompactTest"],
[...]
>           "option": [{
>             "name": "base",
>             "value": "http://example/"
>           }],

Wouldn't it be simpler to have a single JSON object containing all the
options/flags? Something like

  "options": {
    "base": "http://example.com",
    "compactArrays": "true"
  }

I think that would simplify the implementation of test runners as well as
make the test manifests more concise.


> Options are taken from the JsonLdOptions set, except for the RDF tests,
> which have their own option space (produceGeneralizedRDF and
> useNativeTypes)
> 
> Other things we may want to test for error cases:
> * Negative Syntax Tests
> * All raised errors are detected
> 
> Note that it is common for negative tests to not be required for an
> implementation to pass, as these represent reasonable areas of
> extension. We could add a "processingMode" option to each such test set
> to "json-ld-1.0" to override this.

I would say that in the json-ld-1.0 processingMode every processor has to
raise exactly the same errors and return the same error codes to be
conformant. That's exactly the reason why we introduced processing modes.
Otherwise applications on using a processor can't rely on the fact that
errors are reported in a consistent manner.


> Testing the documentLoader option clearly requires a function, but
> could be the function name to be evaluated.

That's indeed a tricky one.. I'm wondering whether we should make that
feature optional!? In any case we need to test remote context resolution.


Thanks for getting this started Gregg


--
Markus Lanthaler
@markuslanthaler

Received on Friday, 23 August 2013 08:52:15 UTC