RE: Test manifest/vocabulary updates

On Saturday, August 24, 2013 11:47 PM, Gregg Kellogg wrote:
> I updated the test-suite instructions with this, and modified the
> vocabulary. The vocabulary's maintained in
> http://json-ld.org/test-suite/vocab.ttl. There is a small ruby 
> script to generate vocab.jsonld and vocab.html (in HTML+RDFa) from
> this. You can see the HTML+RDFa version at
> http://json-ld.org/test-suite/vocab. Between the two documents, developers
> should have everything they need to know to create a test runner to run
> the test suite.

Great work Gregg. I have a couple of comments but I think it's easier to
discuss those things directly in the commits. Thus I added them there [1].


> As the runner should not have any a-priori knowledge of the test
> manifests, we can probably simplify the test organization. I'd like
> to eliminate error-expand-manifest.jsonld and fold that test into the
> expand-manifest.jsonld. It also uses an @type of jld:ApiErrorTest, but
> this is redundant with jld:NegativeEvaluationTest, which already expects
> an error result.

While I fully agree that the runner should not have any a-priori knowledge
of the test manifests, I think keeping them separate simplifies the creation
of test runners a whole lot. You don't need to generalize the test methods
in that case but can keep them very short, a couple of lines in most cases
(here's how I do it [2]). It also makes it much simpler to run just a subset
of the test suite. I would thus prefer to keep them separate.


> Next task is to identify tests; I suggest that we create manifest entries
> for any MUST clause not already having a test (if we were really
ambitions,
> we'd annotate each test with what normative requirement it is testing).

Yeah. I would also suggest to go through the algorithms and write a test for
every error that is thrown.


> If we take a divide and conquer method, we should be able to get through
> these in the next week or so; certainly, prior to entering CR.

I'm quite busy at the moment but I'll certainly find the time to write some.
Just tell me what subset of the spec I should work on.


[1]
https://github.com/json-ld/json-ld.org/commit/73f2c63f0c37f04f46bda17da3f6ac
fe321b282a
[2]
https://github.com/lanthaler/JsonLD/blob/master/Test/JsonLDTestSuiteTest.php
#L69


--
Markus Lanthaler
@markuslanthaler

Received on Sunday, 25 August 2013 11:03:16 UTC