annotation data model -- running test scripts

Shane-
 
We've made progress on rounding out the json schemas / assertions needed for
testing the model.  Still more to generate, but now trying to create and
generate the test scripts to understand better how that part of the process
works.  
 
A. 
One glitch is that the test harness processes assertions (json schemas) as
an entire file, i.e., does not seem to allow reference to individual schemas
within a single .json file.  Can work around this, but wanted to make sure I
wasn't missing something obvious.  So for example, 
 
3.1-annotationMustKeys.json contains 6 sub-schemas in #/definitions/.  So I
wrote the assertions part of the test script thusly:
 
  "assertions": [
 
"annotations/3.1-annotationMustKeys.json#/definitions/annotationContextValid
ated",
 
"annotations/3.1-annotationMustKeys.json#/definitions/annotationIdValidated"
,
 
"annotations/3.1-annotationMustKeys.json#/definitions/annotationTypeValidate
d",
    "annotations/3.1-annotationMustKeys.json#/definitions/notBodyBodyValue",
    "annotations/3.1-annotationMustKeys.json#/definitions/targetKeyFound",
 
"annotations/3.1-annotationMustKeys.json#/definitions/targetObjectsRecognize
d" 
  ]
 
                The test harness ignored the fragment part of the ids.  If
there's a way when specifying assertions to reference sub-schemas directly,
please let me know. Again not critical, but would be nice for a couple of
logistical reasons.
 
B.
Otherwise the tests worked as expected, catching the errors I introduced.
When errors encountered AJV error messages appear in the JSON output from
the test harness.  However, the schemas themselves include errorMessage
values.  These values don't appear in the JSON output as best I can tell.
Do these error messages appear elsewhere, or is there something different I
should be doing to make these messages appear when a schema fails? 
 
C.            
Could you clarify the meaning of the testType parameter?
 
FYI, I will eliminate the old test scripts that reference schemas no longer
present so as to streamline running of the tests. I expect to add several
more manual tests in appropriate folders today. 
 
Thanks,
 
Tim Cole
 
.
 
 
http://testdev.spec-ops.io:8000/annotation-model/annotations/3.1-model-musts
-v2-manual.html