Re: placeholder test files

Further to this discussion and to help illustrate the point, I have added a
couple of things to the WAT repo:

   1. annotations/requiredProperties.json is single assertion that ensures
   all the required properties are present.  It can be referenced from every
   other test to ensure the basic shape is available.
   2. bodiesTargets/3.2.4-textualBody.test shows the beginning of a test
   for textualBody.  It tests the basic requirements and needs to be extended
   with the rest of the optional properties using the same pattern as for that
   of the type property.


On Wed, Jul 20, 2016 at 9:25 AM, Shane McCarron <shane@spec-ops.io> wrote:

> Thanks for this.  Note that, in parallel, I added what I consider complete
> tests for a few things to the WAT tree yesterday as well.  These might be
> massaged to use your updated definitions... I just did the definitions
> inline.
>
> My recommendation would be that you try to not make the tests too
> sequential...  I will be the first to admit that I don't have all of the
> context of the last N years of developing this spec, but I assume if there
> are multiple ways to express body (for example) then it is assumed that
> each of these is important. If that is the case, then:
>
>    1. Have a separate test for each expression type (each feature)
>    2. Make that test as simple as possible
>    3. Don't worry about sequentially determining what to test.  You can
>    test it all at once.
>
> Such an architecture will make it very easy to identify which features are
> supported by multiple implementations.  For example, in the case of body
> and looking at your diagram, to test textualBody have an assertion that:
>
>    - requires body
>    - requires body be an object
>    - require body have a property of value
>    - require body NOT have a property of source nor type nor id
>    - require the value property be a string (?)
>    - then CHECK but do not fail if the optional components are not
>    present (no requirements): language, format, processingLanguage,
>    textDirection
>
>
> That's it.  Have a test like this for EACH form of body.  The instructions
> for the test say supply an annotation that is of the correct form for that
> type.  An implementation is tested by supplying that. Rinse, Repeat.
>
> If you want to see an example of this, check out
> bodiesTargets/3.2.1-bodyValue.test.  That test simply asserts that there is
> a bodyValue that is a string, and that there is NOT a body property.
>
> I do not personally think it is critical that each of these tests also
> ensure that all the required components of an annotation are present.  On
> the other hand, if they are not present the test should not pass.  If
> people feel it is critical those are included as well it is easy enough.  I
> put up a test that does just that in annotations/3.1-model-musts.test.  The
> contents of that could be pulled out into a single .json file and put in
> /common, then referenced from each test as the first assertion.
>
>
> On Wed, Jul 20, 2016 at 8:42 AM, Jacob Jett <jjett2@illinois.edu> wrote:
>
>> Hi Shane,
>>
>> Tim, Janina, and I uploaded a number of new schema definitions for the
>> various body objects yesterday. Our thinking is that the definitions would
>> be used to test the body object and showcase exactly what the object is
>> with respect to the Annotation vocabulary.
>>
>> We've also developed what we think is a good logical flow diagram for the
>> test of the body object (see attached flow diagram). Although in
>> retrospect, we probably want to test the *target* key first since, if
>> it's absent then we already know that the annotation isn't valid or
>> well-formed. We can reuse the attached flow diagram for target objects
>> simply by eliminating the check for the *value* key (i.e., the
>> TextualBody case) from the diagram.
>>
>> Another thing to note about the uploaded definitions--we've adopted a
>> standardized and simple scheme for titles and descriptions.
>>
>> Tim will likely follow up this email later this afternoon with additional
>> details. Will likely discuss some of this on Friday's call.
>>
>> Regards,
>>
>> Jacob
>>
>>
>>
>>
>> _____________________________________________________
>> Jacob Jett
>> Research Assistant
>> Center for Informatics Research in Science and Scholarship
>> School of Information Science
>> University of Illinois at Urbana-Champaign
>> 501 E. Daniel Street, MC-493, Champaign, IL 61820-6211 USA
>> (217) 244-2164
>> jjett2@illinois.edu
>>
>> On Sun, Jul 17, 2016 at 9:30 AM, Shane McCarron <shane@spec-ops.io>
>> wrote:
>>
>>> So... it turns out that the prefix "stub-" is reserved in WPT.  I am not
>>> sure for what, but it is a special class of test files that can't currently
>>> be selected from the UI.  So when I create the empty test files that need
>>> populating, I am going to use the prefix "ph-" for placeholder.
>>>
>>> On a related note, the result of NOTRUN in WPT literally means that a
>>> test was created and then nothing was run about it.  I am going to have the
>>> ph-* tests do this so they show up in our reporting but.... I am sure that
>>> is not what NOTRUN was intended for and we might get some pushback from the
>>> powers that be.  Whoever they are.
>>>
>>> --
>>> Shane McCarron
>>> Projects Manager, Spec-Ops
>>>
>>
>>
>
>
> --
> Shane McCarron
> Projects Manager, Spec-Ops
>



-- 
Shane McCarron
Projects Manager, Spec-Ops