RE: Test manifest/vocabulary updates

On Sunday, August 25, 2013 7:35 PM, Gregg Kellogg wrote:
> It's a bit arbitrary to say that error cases should be treated
> differently than non-error cases; particularly, when we also have
> combinations of errors and options. Already, we'd have to duplicate the
> manifests for error-compact, error-flatten, error-framing, error-
> fromRdf, error-toRdf, in addition to error-expand. That's a lot of
> different test manifests, when each manifest is really just trying to
> test a particular API.

I don't think that's necessary. Almost all errors are raised during
expansion (which includes context processing). There are exactly two errors
that aren't handled by that, i.e., "compaction to list of lists" in the
compaction algorithm and a "conflicting indexes" error in the node map
generation algorithm. Since all the positive evaluation tests already
verified the various algorithms separately, we could simply run all the
error tests using the flattening algorithm which internally uses all the
algorithms. So there really just needs to be one manifest testing the
returned error codes.


> I suggest we continue to use the consolidated
> tests so that, for example, all of the "Expand" functionality is tested
> in an expand-manifest. This would leave expand- compact- flatten-,
> toRdf- and fromRdf-manifests, in addition to framing- and normalize-;
> that seems to me like a reasonable breakdown to keep things separated
> to functional areas. If you would really like to keep error and options
> separate, perhaps a single error-manifest which tests for errors across
> all the API methods (and to/from RDF), and manifests for different
> options. What level of granularity do you feel is important?

Yes, that's what I had in mind. I should have said it right ahead, sorry.


> I'm pretty busy too, unfortunately. We need to look at the following:
> 
> Expand: errors, with options for base, and useDocumentLoader
> Compact: errors, with options for base, compactArrays, expandContext,
> and useDocumentLoader
> Flatten: errors, with options for base, compactArrays, expandContext,
> and useDocumentLoader
> ToRDF: errors, with options for produceGeneralizedRDF
> FromRDF: errors, with options for useNativeTypes
> 
> Perhaps you could work on Expand and/or Compact. Dave might work on
> Flatten, and I could work on To/From RDF.

As said above, almost all errors are raised in the expansion algorithm so we
should try to find another way to split the work.

We have these options (we can ignore processingMode):

 - base
 - compactArrays
 - documentLoader
 - expandContext

I don't think any of those needs more than a 2-3 tests and they don't depend
on each other so there's no state explosion. The documentLoader callback is
also quite simple requiring perhaps a list of a handful IRIs to trigger the
various responses (contextUrl, documentUrl & document or one of the two
error codes). So that could be one work package.

Then there are the errors triggered in the context processing algorithms
which could be a second work package and the errors in the expansion
algorithms which could be a third work package.

Finally I would propose a fourth package dealing with the two errors in
compaction and node map generation and extending the coverage of the RDF
conversion algorithms (there are no errors there).

How does that sound?


--
Markus Lanthaler
@markuslanthaler

Received on Sunday, 25 August 2013 18:39:58 UTC