Re: Conformance Disclaimer -- comments please

I think we should remove the #2. It only adds to the confusion. 
Additionally, recently we discussed the conformance disclaimer in SpecGL 
and removed the paragraph of the disclaimer that talked about not 
satisfying a CP. So, to be consistent - we should remove this requirement 
(i.e., delete #2)

lynne



At 09:32 AM 1/29/2003, Lofton Henderson wrote:

>QAWG,
>
>I have pretty much finished the final WG-review version of OpsGL, for Last 
>Call resolutions.  Since the last WG-discussion draft (20030120), I have 
>been mostly tweaking the wording and adding more "Rationale" sections.
>
>But I have discovered one last clarification issue, and I need your 
>feedback.  OpsGL CP6.4, Conformance Disclaimer.
>
>Two sections follow:  the complete 20021220 text of CP6.4; and, my current 
>(partial) revision.  Following the two sections is my question(s).
>
>### 20021220 text ###
>Checkpoint 6.4. Provide a conformance verification disclaimer with the 
>test materials.  [Priority 1]
>
>To fulfill this checkpoint, the Working Group MUST provide a prominent 
>disclaimer about the use of the test materials for conformance 
>verification of implementations.
>
>Discussion. Although tests suites may be used for conformance 
>verification, the Working Group must make users aware that:
>
>    1. passing all of the tests does not guarantee full conformance of an 
> implementation to the specification
>    2. failing the test suite means failing tests for the specific feature 
> they target
>
>An example of a conformance disclaimer may be found in the Conformance 
>chapter of this specification.
>### end ###
>
>### current editing progress ###
>Checkpoint 6.4. Provide a conformance verification disclaimer with the 
>test materials.
>
>Conformance requirements: the Working Group MUST provide a prominent 
>disclaimer about the use of the test materials for conformance 
>verification of implementations.
>
>Rationale.  It is common to draw unwarranted conclusions about conformance 
>to the specification from test suite results.  A conformance disclaimer 
>clarifies the relationship between test suite results and conformance.
>
>(@@unchanged from here on@@)Discussion. Although tests suites may be used 
>for conformance verification, the Working Group must make users aware that:
>
>1. passing all of the tests does not guarantee full conformance of an 
>implementation to the specification
>2. failing the test suite means failing tests for the specific features 
>they target.
>
>An example of a conformance disclaimer may be found in the Conformance 
>chapter of this specification.
>### end ###
>
>Questions:
>-----
>
>a.) What does #2 mean? (It is hard to parse.)
>
>It seems like "they" really refers to the test suite.  I.e., is the 
>intended statement something like, "Failing the test suite means failing 
>(some?) tests for specific features targeted by the test suite."?
>
>If so... then so what?  What does that say about conformance?
>
>b.)  Are we trying to say (disclaim) something like, "If you fail some 
>tests and therefore fail the test suite, don't try to draw any conclusions 
>beyond the scope of the specific features targeted by the test 
>suite."?  And is that true?!
>
>c.)  Isn't it true that failing one specific-feature test for a MUST 
>requirement of the specification means that the implementation does not 
>conform to the specification?  Maybe that does not sound like 
>"disclaimer", but if it is true, why aren't we saying that?  (Is it too 
>obvious?)
>
>Maybe I'm missing the point altogether, and #a-c are way off.  In any 
>case, if this is clear to you, please comment.
>
>-Lofton.
>
>

Received on Wednesday, 29 January 2003 10:47:30 UTC