Testing API options

There are probably several ways to do this, and Markus may have some other thoughts based on his experience with Hydra, but here is a proposal for updating the test manifest to be able to provide options to API calls.

Currently, a typical test case entry looks like the following:

    {
      "@context": "http://json-ld.org/test-suite/context.jsonld",
      "@id": "",
      "@type": "mf:Manifest",
      "name": "Compaction",
      "description": "JSON-LD compaction tests use object comparison.",
      "baseIri": "http://json-ld.org/test-suite/tests/",
      "sequence": [
        {
          "@id": "#t0001",
          "@type": ["jld:PositiveEvaluationTest", "jld:CompactTest"],
          "name": "drop free-floating nodes",
          "purpose": "Unreferenced nodes not containing properties are dropped",
          "input": "compact-0001-in.jsonld",
          "context": "compact-0001-context.jsonld",
          "expect": "compact-0001-out.jsonld"
        }, {
          ...
        }
      ]
    }

We could create an "option" property, which takes a set of objects to define the option name and value to be passed in. Values are typically strings, but it may be useful to pass in arrays or hashes as values too; however, this would loose an RDF interpretation of the value, so we might not want to go there. This might look like the following:

    {
      "@context": "http://json-ld.org/test-suite/context.jsonld",
      "@id": "",
      "@type": "mf:Manifest",
      "name": "Compaction",
      "description": "JSON-LD compaction tests use object comparison.",
      "baseIri": "http://json-ld.org/test-suite/tests/",
      "sequence": [
        {
          "@id": "#txxx",
          "@type": ["jld:PositiveEvaluationTest", "jld:CompactTest"],
          "name": "Override document base with base option",
          "purpose": "When using the base option, the document location can be modified",
          "option": [{
            "name": "base",
            "value": "http://example/"
          }],
          "input": "compact-xxxx-in.jsonld",
          "context": "compact-xxxx-context.jsonld",
          "expect": "compact-xxxx-out.jsonld"
        }, {
          ...
        }
      ]
    }

Options are taken from the JsonLdOptions set, except for the RDF tests, which have their own option space (produceGeneralizedRDF and useNativeTypes)

Other things we may want to test for error cases:
* Negative Syntax Tests
* All raised errors are detected

Note that it is common for negative tests to not be required for an implementation to pass, as these represent reasonable areas of extension. We could add a "processingMode" option to each such test set to "json-ld-1.0" to override this.

Testing the documentLoader option clearly requires a function, but could be the function name to be evaluated.

Gregg Kellogg
gregg@greggkellogg.net

Received on Thursday, 22 August 2013 23:32:38 UTC