Re: [SpecGL Draft] A.2 Provide the wording for conformance claims.

Le 13 août 2004, à 01:26, david_marston@us.ibm.com a écrit :
> >Provide the wording for conformance claims.
>
> My remarks may overlap with the pro forma item.
>


I don't mind adding the techniques but don't forget too much will kill 
the benefits of SpecGL ***Lite*** and my first problem is that I have 
difficulty to understand them. Trying to rewrite them

> Additional techniques:
> 5. For each discretionary item that is a choice, name the items and 
> the choices. A questionnaire should ask which choice was made by 
> reference to the keywords. (Example: when more than one template 
> matches, do you raise error DE16 ("raise-error") or choose the last 
> template and continue ("choose-last")?)


If there are discretionary items, provide a way to identify them in the 
conformance claim.

> 6. Disclaim, on behalf of the W3C, any public claim of conformance 
> that does not include certain enumerated details of the claim.

	??? This I don't understand.

> 7. Describe how a software buyer can use the pro forma and the 
> requirements for a conformance claim in their own RFQ verbiage.

	The conformance clain is not the proforma. You may have use a Proforma 
to know if you are conformant thought it's not the conformance claim. 
You may want to say that if someone claim the conformance with the 
wording, you may require that the Proforma is added to the claim, but 
that's a separate issue.

> 8. Disclaim any conformance statement that says that less than 100% of 
> the applicable test cases passed, therefore the product is "partially 
> conformant", "X% conformant", "X% passing", "conformant in all but 
> [feature X]", etc. Use the discretionary items, other DoV, and the pro 
> forma to determine the set of "applicable tests" for the product.

8. Forbid the use of any claims which is different of the claim given 
in your specification. examples... X% passing, WG Test suite passing..; 
etc.


> 9. Disavow the notion that the WG-sanctioned test suite is complete 
> and that passing 100% of applicable tests means that the product is 
> (fully) conformant.

9. This is covered by the previous one.

> 10. Disavow the notion that individual test cases in the suite all 
> have equal weight. (Example: the product that fails 20 test cases 
> could be worse than the product that fails 100 cases, depending on 
> which cases and the buyer's needs.)

Not belonging here. It's about the conformance claim, not about passing 
a TS.

> 11. Provide instructions on proper administration of the WG-sanctioned 
> test suite and require that any test reports state what steps were 
> taken where the instructions allow site-specific adaptation. (Example: 
> if actual and expected output are compared in a site-specific way, the 
> source code for the comparitor routine(s) must be published.)

It has nothing to do with the conformance claim, but with the 
implementation report, or Interoperability testing. Which is different. 
I agree you may use one for the other but they are different beasts.


> 12. Provide wording that is usable by a vendor applying the tests to 
> their own product, an assumed-hostile vendor applying the tests to a 
> competing product, and an independent test labs testing several 
> products.

Not belonging to the conformance claim and contradictory with what you 
said previously. The conformance claim is not about applying a test 
suite and reading the results.


-- 
Karl Dubost - http://www.w3.org/People/karl/
W3C Conformance Manager
*** Be Strict To Be Cool ***