Re: Test manifest/vocabulary updates

On Aug 25, 2013, at 11:39 AM, "Markus Lanthaler" <markus.lanthaler@gmx.net> wrote:

> On Sunday, August 25, 2013 7:35 PM, Gregg Kellogg wrote:
>> It's a bit arbitrary to say that error cases should be treated
>> differently than non-error cases; particularly, when we also have
>> combinations of errors and options. Already, we'd have to duplicate the
>> manifests for error-compact, error-flatten, error-framing, error-
>> fromRdf, error-toRdf, in addition to error-expand. That's a lot of
>> different test manifests, when each manifest is really just trying to
>> test a particular API.
> 
> I don't think that's necessary. Almost all errors are raised during
> expansion (which includes context processing). There are exactly two errors
> that aren't handled by that, i.e., "compaction to list of lists" in the
> compaction algorithm and a "conflicting indexes" error in the node map
> generation algorithm. Since all the positive evaluation tests already
> verified the various algorithms separately, we could simply run all the
> error tests using the flattening algorithm which internally uses all the
> algorithms. So there really just needs to be one manifest testing the
> returned error codes.
> 
> 
>> I suggest we continue to use the consolidated
>> tests so that, for example, all of the "Expand" functionality is tested
>> in an expand-manifest. This would leave expand- compact- flatten-,
>> toRdf- and fromRdf-manifests, in addition to framing- and normalize-;
>> that seems to me like a reasonable breakdown to keep things separated
>> to functional areas. If you would really like to keep error and options
>> separate, perhaps a single error-manifest which tests for errors across
>> all the API methods (and to/from RDF), and manifests for different
>> options. What level of granularity do you feel is important?
> 
> Yes, that's what I had in mind. I should have said it right ahead, sorry.

Okay, I'll revert the change and turn error-expand-manifest into just error-manifest.

>> I'm pretty busy too, unfortunately. We need to look at the following:
>> 
>> Expand: errors, with options for base, and useDocumentLoader
>> Compact: errors, with options for base, compactArrays, expandContext,
>> and useDocumentLoader
>> Flatten: errors, with options for base, compactArrays, expandContext,
>> and useDocumentLoader
>> ToRDF: errors, with options for produceGeneralizedRDF
>> FromRDF: errors, with options for useNativeTypes
>> 
>> Perhaps you could work on Expand and/or Compact. Dave might work on
>> Flatten, and I could work on To/From RDF.
> 
> As said above, almost all errors are raised in the expansion algorithm so we
> should try to find another way to split the work.
> 
> We have these options (we can ignore processingMode):
> 
> - base
> - compactArrays
> - documentLoader
> - expandContext
> 
> I don't think any of those needs more than a 2-3 tests and they don't depend
> on each other so there's no state explosion. The documentLoader callback is
> also quite simple requiring perhaps a list of a handful IRIs to trigger the
> various responses (contextUrl, documentUrl & document or one of the two
> error codes). So that could be one work package.
> 
> Then there are the errors triggered in the context processing algorithms
> which could be a second work package and the errors in the expansion
> algorithms which could be a third work package.
> 
> Finally I would propose a fourth package dealing with the two errors in
> compaction and node map generation and extending the coverage of the RDF
> conversion algorithms (there are no errors there).
> 
> How does that sound?

Sounds good to me, I'll see if I can get started on some relating to the context processing today.

I would just add options testing to the existing per-method tests; I'll do some of those too.

Gregg

> --
> Markus Lanthaler
> @markuslanthaler
> 
>

Received on Sunday, 25 August 2013 19:54:27 UTC