Re: Framework documents nature

At 10:01 AM 1/8/02 -0700, Lofton Henderson wrote:
>At 10:11 AM 1/8/02 -0500, Mark Skall wrote:
>>At 08:33 AM 1/8/02 +0100, Daniel Dardailler wrote:
>>
>>> > Coming from a conformance background, I'm still disturbed by providing
>>> > recommendations rather than requirements when the implications of not
>>> > following our processes could have a catastrophic impact on
>>> > interoperability and the quality of implementations.  If we believe 
>>> what we
>>> > say about the importance of our activity, and we don't require many 
>>> of the
>>> > things we're asking for, then, by our own admission, we are inviting 
>>> disaster.
>>>
>>>I think we should have the same approach as for the WAI guidelines
>>>(Lynne seems to agree with me here): our QA framework specifies
>>>requirements, with various level of importance (MUST, SHOULD, MAY, P1,
>>>P2, A, AA, etc), and someone else decides what to do with them.
>>>
>>>This someone else can be the W3C Advisory Board stating that all WGs
>>>must comply with the QA Framework Spec Guidelines to level AA, or with
>>>the QA Framework Process Guidelines level A.
>>
>>
>>I agree.  The WAI guidelines are an excellent model.  It seemed to me 
>>that, based on our discussions so far, we were going to fall WAI (pardon 
>>the pun) short of that.
>
>Could you be more specific?  It is my belief that QA intends to follow the 
>WAI model as closely as possible, deviating only when it is sensible to do 
>so (Ian pointed out a couple of possible such instances).  If we are 
>missing the mark (pardon the pun), it would help to have details so that 
>we can adjust our aim.


Okay, here goes.  The way (or is it wai) I understand the WAI guidelines it 
that there are three different types of checkpoints - priority 1, 2 and 
3.  Priority 1 means this "must" be adhered to; priority 2 "should" be 
adhered to; priority 3 "may" be adhered to.

There are then 3 conformance levels - A (all priority 1 checkpoints 
satisfied); Double A - priority 1 and 2 checkpoints satisfied; Triple A - 
priority 1, 2 and 3 checkpoints satisfied.

This scheme is pretty constraining.  It explicitly defines 3 different 
types of conformance and what is required to be able to claim these.  To 
me, these are requirements.  You can never force someone or a WG to do what 
we ask but (if we adopt this model)  you've "forced" a WG to do what we ask 
if they want to say they're conforming.  Notice, to claim even the weakest 
type of conformance, one must satisfy all priority 1 (must) 
checkpoints.  This is completely analogous (in my opinion) to putting 
"shall" or "must" requirements on an implementation when writing a 
functional standard.  You can never "force" an implementer to do anything, 
but if he/she wants to claim they conform, then they had to have met 
all  the requirements.

Again, this is quite different than saying that the framework is "nominally 
informative" - this tells me that nothing is required.


>I have also been under the impression, since Brussels, that our approach 
>to hard requirements (e.g., Process Document modifications) is as Daniel 
>said above, "...and someone else decides what to do with them."
>
>>My main point is that we need to document the issue, have a discussion 
>>and come to a resolution.  I just wanted to make sure that we have a 
>>cohesive strategy that is consistent with how we see our documents being used.
>
>Per previous message, it is partially dealt with in Issue #16, which was 
>discussed at Brussels and more or less closed.
>
>But perhaps Issue#16 is too narrowly stated to capture your concern.  In 
>that case, would you mind suggesting modifications to the wording of #16 
>that would accurately express your concern?  Then we can reopen it for 
>further discussion and a definitive closure.


How about "Shall the QA framework documents be normative or 
informative?"  Or "Shall the QA framework documents impose specific 
requirements on the W3C Working Groups that must be satisfied before they 
can claim conformance?"



>Note also that this is affected by Issue#39, 
>http://www.w3.org/QA/WG/qawg-issues-html.html#x39, SHALL, SHOULD, MAY 
>usage.  Because our approach might be to carry the priority in the 
>conformance keywords.  (This is tagged as "editorial", but it really is 
>substantive -- what is the normative force and priority of each checkpoint).
>
>-Lofton.
>
>
>-Lofton.
>

****************************************************************
Mark Skall
Chief, Software Diagnostics and Conformance Testing Division
Information Technology Laboratory
National Institute of Standards and Technology (NIST)
100 Bureau Drive, Stop 8970
Gaithersburg, MD 20899-8970

Voice: 301-975-3262
Fax:   301-590-9174
Email: skall@nist.gov
****************************************************************

Received on Tuesday, 8 January 2002 17:59:01 UTC