W3C home > Mailing lists > Public > www-qa-wg@w3.org > December 2002

Re: [www-qa-wg] <none>

From: Lofton Henderson <lofton@rockynet.com>
Date: Sun, 01 Dec 2002 21:17:22 -0700
Message-Id: <>
To: www-qa-wg@w3.org

Here are some thoughts and opinions on the SpecGL topics for Monday...

At 12:34 PM 11/29/02 -0500, lsr@nist.gov wrote:
>IMPORTANT:  There are 2 aspects to consider here:
>1. changes that need to be made to the CP to clarify the CP. (labeled For CP)
>2. changes we need to make to the SpecGL so that it conforms to the SpecGL
>(based on Mark’s review [1]).  (Labeled, For SpecGL)
>General (applies to many CPs):  When a CP doesn’t apply, should a
>specification be required to indicate somewhere, that xxx doesn’t 
>apply?  This
>could be done in the ICS. Should CPs include a discussion of what happens if
>the CP doesn’t apply?

This is a tricky question. We should look at those ckpts that seem 
difficult to apply to a GL document (esp. SpecGL), and ask "why?" (i.e., 
why is this CP difficult to apply to SpecGL?). Some possible answers to the 
"why?" question:

** we misunderstand the nature of GL specs;

** or, we are not abstracting their properties correctly;

** or, some ckpts are inherently N/A to something like SpecGL (in which 
case, we might need a generalized "N/A" escape clause -- we may have failed 
to anticipate and appreciate other N/A situations);

** or, we have worded the ckpt incorrectly so that it only *appears* to be N/A.

>2.1 Identify all classes of products
>What are the classes of products?  According to the Scope, the SpecGL applies
>to Specification. However, we go on to distinguish between existing specs and
>new specs.  Also Conformance Clause addresses WG specifications.
>What is the class of products?  1 class= specification or 2 classes: new
>specs and old specs.
>Clarification:  In the scope we define specifications to be W3C Technical
>supports.  Do Technical Reports include Recommendations and Guidelines or
>just Recommendations?

My interpretation.  Scope includes and is limited to new Technical 
Reports.  The reference to "WG specifications" in the Conformance section 
is carelessness (we fail the "consistent terminology" CP!).  From a W3C 
Process perspective, there is nothing called "Guidelines".  In WAI, the 
guidelines documents are Recommendations (or on the way to being Recs).

There is one class:  new TRs.  The talk about old specifications is 
informative discussion.  Or more precisely, "old TRs are out of scope", 
although it may be useful to look at them from the perspective of SpecGL in 
order to understand their conformance characteristics, strengths and 
weaknesses (e.g., to fix problems in their next edition or version).

>For SpecGL, specifically state in the Scope, the class of product that this
>document targets is xxxx.

I agree that we should be more careful to define class of product (CoP) and 
category of SpecGL, in SpecGL.  I disagree with Mark's assertion that we 
necessarily need to add "specification" to the enumerated list(s).  It 
depends on how widely useful a "specification" CoP is.  Is SpecGL likely to 
be the only (Rec, or Note, or whatever) that addresses the "specification" 
CoP?  If so, then we should not add it to the enumerated list.  We should 
use the "...else define your CoP" variation of the requirement.

>13.2 Distinguish normative and informative text. (David and Mark listed this)
>SpecGL specifically identifies non-normative references, but otherwise it
>isn’t clear.  Is it sufficient to only label informative items and by
>default everything else is normative? Do we want editors to label all
>sections, explain in Scope what is or isn’t normative?
>Proposal: For CP, do we need to add anything here or cover this in ExTech?

I think what we need is a better common understanding of what we mean by 
normative and informative.  A fairly stringent view would be -- (almost) 
the only normative bits are the test assertions.  I.e., in SpecGL, the bits 
in the "to fulfill" sections (which are marked-up as class="TA", i.e., 
these are the test assertions).

I said "almost", because I guess stuff like the ch.3 Conformance section 
are normative also.  But in the GL and the CP text, it is the test 
assertions that are clearly normative.  Since these are somewhat terse and 
abbreviated, they are often followed by supplemental clarifying 
verbiage.  This is the tough stuff to decide -- is that verbiage 
informative, or is it normative?

>For SpecGL, in Scope add a general statement that examples and the
>rationale are informative unless otherwise indicated and that the GL, CP,
>and ‘to fulfill…’ are normative.

IMO, this is the right approach.

If that is agreed, then we need to sort out whether we consider the "gray 
stuff" -- clarification of terse normative language and test requirements 
-- to be normative or informative.

>Is the ExTech a normative or informative reference?  (see also 10.3 below)
>Proposal: For SpecGL, add it to the appropriate reference list?

Extech is entirely informative.  OpsET (OpsGL Extech) is fairly explicit 
about this -- no conformance requirements, all conformance requirements are 
in OpsGL -- although it is not explicitly stated in OpsGL.

>10.3 Make normative reference to specifications on which the current
>specification depends.
>Does the SpecGL depend on the ExTech?  It does to satisfy CP 1.3 Provide
>examples.  So, is ExTech normative or informative?


>14.1 Provide test assertions (for SpecGL conformance to SpecGL)
>Issue 99: Scope of definition of test assertion.
>What are the test assertions for the SpecGL?  The CP or the ‘to
>fulfill…’?  Is it ‘derived from the spec’s requirements? Can they all be
>measures/tested? (e.t., 13.3). Test assertion defined as a statement of
>behavior, action or condition that can be measured or tested.  It is
>derived from the specification’s requirements.
>Issue 99: Does the definition preclude expressing a test assertion in a
>formal language without words and phrases rather than in sentences?  What
>is definition in Glossary?
>Is there a mapping between the spec and test assertion?
>Proposal:  ?

We have clearly stated in Issue List resolutions that the CP statement is a 
"title" for the CP.  The normative requirements -- i.e., the test 
assertions -- are contained within the body of the CP verbiage.  SpecGL and 
OpsGL have isolated the test assertions into the "to fulfill" markup 
sections, using RFC2119 terminology.  Those are the test assertions of 
those two GL documents.

I agree that there are some document conventions and verbiage clarification 
to be addressed here, but the parts (i.e., the TAs) are in place and even 
marked up as such (class="TA").

>12.1 Provide an Implementation Conformance Statement proforma
>Is the Checklist for the SpecGL the ICS?
>Proposal; use the Checklist as the ICS, make necessary adjustments.  Editor
>(and Lofton?) action.

The checklist is an ICS.  Although it is not labelled as 
such.  Question.  Is an ICS not only required to be present, but also 
required to be labelled as "ICS"?  I.e., the requirement is "...MUST have 
an ICS" (SpecGL meets this).  Should it say instead, "...MUST have an ICS, 
and SHOULD label it as an ICS"?

>1.4  Provide an example for each formal description
>What is a formal description? Definition is unclear.
>Does SpecGL provide this?

Sandra suggested a definition and it is now in CP 1.4.

It should be added to section 6, "Definitions".

>13. 3 Use consistent terminology
>Can this be verified?  What exactly is required? Should this CP be removed
>and left to Manual of Style and Susan L to worry about?

No, because Style Manual conformance is not required for publication.

Let's try to make it verifiable.  "...MUST NOT use different terms or 
phrases for the same concept [ed.  w/o an accompanying specific reference 
to the principal term or phrase?]"

>GL8: Discretionary Items and
>13.4 Provide a fast way to find conformance information (David and Sandra
>Issues of navigation and discovery were under-represented in the
>checkpoints.  For example, discretionary items are only detectable by a
>thorough reading of the entire document.
>Proposal: ?
>1.1 Define the scope of the specification
>Clearly there is a scope section.  But much of the wording is vague and
>imprecise, e.g., ‘it is not expected…’
>Proposal: editor action to clean up
>CP 1.2 Provide User Scenarios
>CP 1.3 Provide examples
>Proposal (Karl): For CP change to Use Scenarios
>For SpecGL: Include in ExTech, link is already provided.

I doubt ET will have User Scenarios (or Use Scenarios).  This was an issue 
a little while ago.  When I rewrote the Introduction, I had a placeholder 
subsection for "User Scenarios", with the disclaimer that there was no time 
to write it before 8-nov publication.  So we decided to drop it for 8-nov 
and complete it later.

In other words, it is to-be-done.

>2.3 Identify which of the categories of object are specified in the
>document as a whole
>Unclear what ‘types of objects’ are and what object SpecGL would be.  List
>given in GL 2 are called ‘categories’.  Also need to define it, not just
>give an example of what it is.
>Proposal: For CP, change ‘types of objects’ to ‘categories’, also add
>category of ‘guideline’ to the list.

I don't think we need to necessarily add to the list.  It is acceptable for 
SpecGL to define its own Category and CoP(s).  So let's decide if we want 
to add a class-of-one to the lists, or take the latter (self-define) 
approach.  I favor the latter.

>3.1 Specify any universal requirements for minimum functionality
>Unclear what is meant by this requirements.  For the SpecGL, is Level A
>considered the minimum functionality?  Does a specification (in our case
>SpecGL) need to explicitly say ‘Level A is the minimal requirement for all
>specifications’ .

I think that is sensible -- we intend that all specs should achieve at 
least degree A conformance.

>3.2 Identify strict conformance requirements
>SpecGL doesn’t explicitly specify this, although it is implied that SpecGL
>doesn’t do strict conformance (see conformance claim wording section 3.2, “
>…meets at least all degree X conformance requirements’.
>9.1 Indicate if extensions are disallowed
>9.2 Indicate if extensions are allowed
>Is there a way to combine these?
>Proposal: For CP, Lofton proposal [find email]
>For SpecGL, need to add explicit words to address the itemized list.  Not
>have use cases or project requirements, so can’t do last MUST. Editor action
>   9.3 Prevent extensions from contradicting the specification
>Proposal: For SpecGL, Editor action to explicitly state.

(...strict conformance, extensions ... this is tricky for SpecGL.  I don't 
have a clear understanding of it yet.)

>11.2 Provide specific wording of the claim
>Make sure all MUST items are in included in Section 3.2.
>Should the example at end of Section 3.2 move to ExTech

No, I think it is useful here.  We do include short examples in SpecGL 
where they help comprehension.  SpecET is for more extensive examples and 
possible strategies.

>For SpecGL conformance to SpecGL, not all the items are satisfied.
>Proposal: For CP and For SpecGL, Editor action in Section 3.2, add version
>and date to conformance claim. Also add them to the list.

Btw, I disagree with Mark's comment that specific wording is not 
provided.  Looking at the 3 bullets, the specific wording is what follows 
the first COLON on each bullet.  Lack of specific wording is not why SpecGL 
fail the checkpoint.  It fails because it omits some required bits, which 
Mark pointed out -- it need more bullets (and wording), to 
require:  identification of the subject of claim, version of SpecGL, and 
date of claim.

Received on Sunday, 1 December 2002 23:21:28 UTC

This archive was generated by hypermail 2.3.1 : Tuesday, 6 January 2015 21:14:29 UTC