Re: Thursday's Discussion

At 11:52 AM 12/14/03 -0500, Mark Skall wrote:

>On Thursday, we will go over the test assertions that I generated for 
>SpecGL.  One of the main things I want to focus on is whether the 
>conformance requirements need to be made more specific.  For some 
>checkpoints I had to rely on the Discussion, Rationale or Example sections 
>to get more information or to get more detail (e.g., conformance 
>requirements for Chkpt 7.2 requires to distinguish normative text from 
>informative content.  The discussion adds normative and informative 
>examples, illustrations and use cases as well.)
>
>There are 2 ways to proceed:
>
>1) In the future, we could ignore these sections and just generated the 
>assertions from the conformance requirements.  However,  for SpecGL, I 
>don't think the test assertions would have been specific enough to 
>generate tests;

This is a side issue, but related -- if we are going to look at evaluation 
criteria such as "specific enough to generate tests", it helps to have a 
concept of what tests for SpecGL might look like.

What do you think of when you think of "tests"?

As we discussed earlier (but I'm not sure of our level of consensus), a 
manual test could be as simple as a per-conformance-requirement (or per-TA) 
question, "Does the spec ...blah...?  Choice of:  NO; YES at 
[...fill-in-link-to-spec-loc...]"  "Blah" is effectively the statement of 
the conformance requirement.

Are you thinking of manual testing like this?  Or are you thinking of some 
sort of automated SpecGL tester?  Or ...?


>2) We could clarify and expand the conformance requirements 
>to  incorporate the concepts in the Discussion, Rationale and Examples 
>section.  This would be my preferred choice.
>
>If we choose 2) we need to set aside extra time for a feedback loop after 
>the assertions are generated in order to rewrite the conformance requirements.
>
>Please look over the test assertions before Thursday's call and be 
>prepared to discussion ones that you feel are contentious.

Can you also supply a list of the ones that you found to be problematic 
(and why)?

-Lofton.

Received on Monday, 15 December 2003 11:53:07 UTC