Re: DoV and verifiability

Alex,

As I'm nearly out of time before a week of travel, my replies are 
abbreviated.  In general I'm avoiding the DoV or no-DoV (bottomless) issue.

At 11:19 AM 9/6/02 -0600, Alex Rousskov wrote:
>Your own explanations contradict that there is an uncertainty. You
>(and QAWG) seem to be certain that the "minimal set" requirement is a
>good valid SHOULD.

There has be no dissent on those bits of SpecGL so far.

>The uncertainty seems to be related to whether we
>should go beyond that simple and generic requirement:

Thanks for the correction.

>[...]
> > Also, do you distinguish between establishing (in SpecGL) the
> > parameters (dimensions) as organizing points of such analyses, on
> > the one hand, and the results of such analyses (somewhere) on the
> > other?
>
>I think both do not belong to SpecGL.

Strongly disagree about the first, the enumeration and definition in SpecGL 
of the DoV (or some suitable DoV, if the present DoV enumeration is decided 
to need improvement).

(Unless we are misunderstanding again -- I'm talking about defining and 
enumerating the major DoV in SpecGL.  I feel less strongly about the 
current "negative disclaimer" requirement in SpecGL [CK10.5], which is a 
requirement that SpecGL places on target specs, and perhaps that is what 
you're reading into the above?)

>There is no need to establish a
>parameter that the spec itself is not using. This would violate,
>again, the "minimum set" requirement.

Disagree again.  (Unless again, I'm talking about definition and 
enumeration of the DoV in SpecGL, and you're talking about the topic of 
addressing each dimension of DoV in target specs, whether or not that 
dimension is a factor in that spec).

>[...]
> > Checkpoint X.Y ...blah...
> >
> > * (requirement)...MUST...
> > * (requirement)...SHOULD...
> > * (requirement)...SHOULD...
>
>Yes. To follow our own guidelines, I would argue that the requirements
>should be numbered/IDed.

This is planned.  In fact, they will be tagged with a granular grammar 
(XMLspec or XHTML+).

>Checkpoint itself is just a subsection
>structure of no normative importance. You do not satisfy/test
>Checkpoints, you satisfy/test requirements. There is no need to
>introduce a second normative level, IMO. Keep it simple and flat.

Experience so far is that this will lead to many more checkpoints.  This 
experience is corroborated by the WAI documents.


> > If there is something that (we think) is worth saying, and if we
> > find that it is "virtually impossible to objectively prove", then
> > why not put it as a SHOULD requirement?  Not matter what a spec
> > writer does, that requirement will not lead to failure of the
> > checkpoint as a whole.
> >
> > If that is true, what is the value?  There is a school of thought that a
> > lot of the value of SpecGL will be in simply helping editors (and WGs) to
> > focus on the key issues and make explicit, defensible choices.  I agree
> > with it, having seen (for each of the various of the DoV)  examples in WGs
> > where the outcome was somewhat default, and probably not carefully
> > considered.  (In fact, I confess -- I'm guilty of some of that.)
>
>Agreed, except I think _we_ should not look at SpecGL as a helping
>guide. That is, SpecGL should apply to a given spec, not to the
>process of writing specs. Other spec authors will use it as they
>please, and yes, it will be helpful as a guide (as a side-effect).

Do you have any specific pointers into SpecGL where we tell the author how 
to write the spec, i.e., what to do?  (Other than a little such unintended 
language in GL14).  When I think of dictating process or telling how to do 
it, I think of, for example, "get out your Big Chief pad and #2 pencil, get 
coffee or beverage of choice, sit comfortably".

Triviality aside, perhaps I'm too close to it, and am forgetting some 
places where we dictate the process as opposed to the result (i.e., what 
should be in the spec).

Perhaps it is the language of the checkpoints' statements:  "Define the 
scope of the specification" sounds like a process imperative to the 
author.  But its explanation clearly describes a condition on the result.

Maybe my choice of words is confusing the issue.  By "helping guide", I do 
not mean to imply a step-by-step guide to writing a specification.  I 
mean:  constraints on the result help me (the author) to figure out what I 
should produce, and in that sense it is a "helping guide".  In other words, 
it is a "side effect" of authors having to meet constraints on the spec 
(but one which, IMO, is as valuable as any other effect of SpecGL).

> > >         - Is it appropriate for SpecGL to illustrate a "minimal
> > >           set" SHOULD with a lengthy discussion about some common
> > >           degrees of variability?
> >
> > You're losing me here.  Can you point to an example in SpecGL?
>
>All sections that talk about DoV are, essentially, an
>examples/illustration of the "minimal set" SHOULD. You said it
>yourself above: QAWG wants to split a generic SHOULD into a number of
>more specific SHOULDs using DoV framework.

A concrete SpecGL pointer or quotation would help.

"Split"?  No, that is not my intention (I shouldn't pretend to speak for 
QAWG on this).  My intention is that SpecGL should contain:

-- the generic SHOULD:  "if a spec can cover the same technology without 
using variation X, it SHOULD be written without X"

-- plus, *additional* more detailed requirements:  if variation X is 
used/allowed in the specification then CKX.1 ... CKX.n give additional 
MUST/SHOULD/MAY requirements about its (X's) use.

This is exactly what is in SpecGL now for each of (or most of) the 
dimensions of DoV, GL3-GL9 (i.e., each of the "X"):  the generic SHOULD 
(okay, disguised as a "Caveat:" with somewhat timid wording), along with 
additional more detailed requirements.

>[...]
>The presence of DoV model _itself_ violates the "minimal set"
>SHOULD.

Do you mean the presence of the DoV model as a feature of the way SpecGL is 
written?  Or do you mean the presence and reference to the DoV model in the 
* target spec* itself, as opposed to in the SpecGL?  (If the latter, then 
we have been arguing about different things.)

[...cut lots of "to-DOV or not-to-DoV" discussion...]

>[...]
> > Just to make sure I'm understanding you, can you point me to an
> > example in SpecGL text of what you mean by "instructions how to
> > write a [good] spec"?
>
>Everything in the form "the authors should consider or be aware of
>this or that" is an instruction to the authors.

I have searched SpecGL for "author" and found about 19 occurrences.  There 
are zero usages such as you quote.  Except for a couple places in GL14, 
where I agree the wording should be modified to pertain to the result ("use 
a grammar to author the specification" is easily transformed into a 
requirement on the specification itself.)

>The entire DoV model
>is only useful if you are writing a spec and looking for instructions
>on how to do it and what to avoid. IMO, if SpecGL is a spec, it should
>be about other specs (the subject is a spec), not about ways of
>writing other specs (the subject is the author of a spec or the
>writing process).

I think the vast preponderance of the language is exactly about the result 
-- to what constraints specs should/must conform.  I have a hard time 
finding more than a couple deviations on this.


> > As I guess I have made clear, IMO the DoV is a useful approach for
> > SpecGL.  I agree that your "'minimal set' SHOULD" ought to be part
> > of SpecGL.  I go further and think that other SHOULDs are useful and
> > helpful to spec editors, even if they are not objectively verifiable
> > (and won't lead to failure of a checkpoint, because they are
> > SHOULD).  And I'm sure that we can and should tighten the prose, and
> > probably even trim and tighten the particular collection of
> > guidelines and checkpoints.
>
>I understand your position. IMO, "useful and helpful [instructions] to
>spec editors" should be outside of SpecGL scope. I suggest that, given
>a spec S and SpecGL, it would be possible to assert that, with a good
>probability,
>         - S is AAA conformant (all SpecGL MUSTs and SHOULDs are satisfied)**
>         - S is AA conformant (all SpecGL MUSTs are satisfied)
>         - S is not conformant (at least one SpecGL MUST is violated)
>AND that
>         - SpecGL is AAA self-conformant
>
>Such a SpecGL would be compact and very useful, especially if there
>are a bunch of free-style tutorials to complement it. Tutorials should
>talk about how to accomplish AAA conformance (which would, in some
>cases, involve handing many DoV issues).

I tend to get lost in long shaggy email dialogs without examples, and I'm 
not sure that I completely understand what your ideal SpecGL looks 
like.  How about an outline?  A table of contents with just the briefest of 
indications what GL and CK you would keep, which ones you would throw out 
(new ones you would add).  And/or you could a Checklist table similar to 
[1].  Or ...

**Footnote.  We considered the model:  AA = MUST+SHOULD, A = MUST, etc.  We 
decided instead on the current WAI model:  prioritized checkpoints (P1, P2, 
P3):  A=P1, AA=P1+P2, AAA=P1+P2+P3.  Multiple requirements can be bound 
within a checkpoint, and we think (but QAWG has not definitively decided) 
that they may carry different normative force (MUST or SHOULD).

Regards,
-Lofton.

[1] http://www.w3.org/TR/qaframe-spec/qaframe-spec-checklist.html

Received on Friday, 6 September 2002 18:42:30 UTC