Re: Test manifest/vocabulary updates

Gregg Kellogg

On Aug 25, 2013, at 12:53 PM, Gregg Kellogg <> wrote:

> On Aug 25, 2013, at 11:39 AM, "Markus Lanthaler" <> wrote:
>> On Sunday, August 25, 2013 7:35 PM, Gregg Kellogg wrote:
>>> It's a bit arbitrary to say that error cases should be treated
>>> differently than non-error cases; particularly, when we also have
>>> combinations of errors and options. Already, we'd have to duplicate the
>>> manifests for error-compact, error-flatten, error-framing, error-
>>> fromRdf, error-toRdf, in addition to error-expand. That's a lot of
>>> different test manifests, when each manifest is really just trying to
>>> test a particular API.
>> I don't think that's necessary. Almost all errors are raised during
>> expansion (which includes context processing). There are exactly two errors
>> that aren't handled by that, i.e., "compaction to list of lists" in the
>> compaction algorithm and a "conflicting indexes" error in the node map
>> generation algorithm. Since all the positive evaluation tests already
>> verified the various algorithms separately, we could simply run all the
>> error tests using the flattening algorithm which internally uses all the
>> algorithms. So there really just needs to be one manifest testing the
>> returned error codes.
>>> I suggest we continue to use the consolidated
>>> tests so that, for example, all of the "Expand" functionality is tested
>>> in an expand-manifest. This would leave expand- compact- flatten-,
>>> toRdf- and fromRdf-manifests, in addition to framing- and normalize-;
>>> that seems to me like a reasonable breakdown to keep things separated
>>> to functional areas. If you would really like to keep error and options
>>> separate, perhaps a single error-manifest which tests for errors across
>>> all the API methods (and to/from RDF), and manifests for different
>>> options. What level of granularity do you feel is important?
>> Yes, that's what I had in mind. I should have said it right ahead, sorry.
> Okay, I'll revert the change and turn error-expand-manifest into just error-manifest.
>>> I'm pretty busy too, unfortunately. We need to look at the following:
>>> Expand: errors, with options for base, and useDocumentLoader
>>> Compact: errors, with options for base, compactArrays, expandContext,
>>> and useDocumentLoader
>>> Flatten: errors, with options for base, compactArrays, expandContext,
>>> and useDocumentLoader
>>> ToRDF: errors, with options for produceGeneralizedRDF
>>> FromRDF: errors, with options for useNativeTypes
>>> Perhaps you could work on Expand and/or Compact. Dave might work on
>>> Flatten, and I could work on To/From RDF.
>> As said above, almost all errors are raised in the expansion algorithm so we
>> should try to find another way to split the work.
>> We have these options (we can ignore processingMode):
>> - base
>> - compactArrays
>> - documentLoader
>> - expandContext
>> I don't think any of those needs more than a 2-3 tests and they don't depend
>> on each other so there's no state explosion. The documentLoader callback is
>> also quite simple requiring perhaps a list of a handful IRIs to trigger the
>> various responses (contextUrl, documentUrl & document or one of the two
>> error codes). So that could be one work package.
>> Then there are the errors triggered in the context processing algorithms
>> which could be a second work package and the errors in the expansion
>> algorithms which could be a third work package.
>> Finally I would propose a fourth package dealing with the two errors in
>> compaction and node map generation and extending the coverage of the RDF
>> conversion algorithms (there are no errors there).
>> How does that sound?
> Sounds good to me, I'll see if I can get started on some relating to the context processing today.
> I would just add options testing to the existing per-method tests; I'll do some of those too.

I changed error-expand- to just error- and added 22 new test cases covering all the specified errors in the expansion algorithms: "Context Processing Algorithm", "Create Term Definition", and "IRI Expansion".


> Gregg
>> --
>> Markus Lanthaler
>> @markuslanthaler

Received on Monday, 26 August 2013 05:53:51 UTC