Draft minutes from QAWG Reading F2F, Oct 28 afternoon

DRAFT

QAWG Face-to-Face meeting minutes
The Open Group, Reading, UK
Thursday 28 October, afternoon


Topic: review of SpecGL

Discussion of Principles that seem to be non-testable

4.3 Principle C - Prevent extensions from breaking conformance

[LR] Why is this principle still marked as "under discussion"?
[KD] Because we haven't fleshed it out
[LR] Rewording could make this clearer
[KD] TAG pointed out that spec can *allow* extensions, so by definition 
they do not break conformance
[MS] Whether or not we agree with this, the requirement is non-testable
[LH] Why are we discussing testability?
[DH] Explains
[LH] But we said months ago that we don't care too much about 
testability - "good faith" is OK
[DH] Even so, some simple rewording can make it easier to determine 
whether one has conformed
[MS] We still need unambiguous language
[LH] Agrees
[MS] What are we trying to say?
[LR] It's impossible for spec writers to prevent those who create 
extensions from breaking conformance.
However, this can be discouraged by the spec-writers, particularly in 
their conformance clause.
[KD] Suggests a rewording: If you define an extension it must conform to 
the rules defined for extensions.
[PC] The intent is to prevent 'well-formed' or 'legal' extension from 
'overriding' something defined in
the original specification. (If spec says "if x then y", you must not 
create a legal extension that results
in "if x then z"
[KH] Provides an example from CSS3 - background-color property could be 
legally 'redefined' by
creating a new -moz-background-color, but spec could require this should 
not modify the defined behaviour
[DH] Such requirement should be spelled out in the definition of the 
extensibility mechanism itself
[PC] Reword Principle to "The definition of the extensibility mechanism 
must state a requirement
that extensions not break compatibility" (exlain "break" in 'What it means')
[MS] What we are asking the spec-writers to do is testable, but what we 
want them to ask their
implementors to do is not testable.
[PC] A warning is still valuable. Could change wording to "must include 
a warning that extensions
should not break compatibilty".
[LR] Need we say that extension mechanisms should not in themselves 
break conformance?
[PC] This is unnecessary - we already require internal consistency

[AI] LR will rewrite this (due Nov 5)
[AI] KD will review (due Nov 8)


Moving on to KD's issues (not documented in advance)
[DH also has some issues, also not circulated in advance]

[KD] "Conformance to this document/Conformance Claims/Examples"
Do we need template (of someone conforming to SpecGL)?
Yes.

[AI] KD will draft a template (due Nov 5)


[KD] Picking up LH's comments from today's agenda:
reading-010
Lofton - [[[1.1 Good Practice C Specify in the conformance clause how  
to distinguish normative from informative content ]]]
[[[ There's still an issue from Lofton if we should define here the way 
the normative language is defined. ]]]
http://lists.w3.org/Archives/Public/www-qa-wg/2004Aug/0081.html

This requirement has disappeared from an earlier draft: "In the 
conformance clause define
how normative language is expressed"

[DH] Asks LH to explain why this is a problem.
[LR] The concept has disappeared altogether

[all] We have two sections addressing this: 1.1.C and 3.2.A
[LH] Everything important about conformance should be accessible 
(linked) from the Conformance Clause section
[DH] Is it OK to make this a Technique?
[LH] OK - if "strongly recommended"...

[Conclusion] 'Technique' text 3.2.A will strongly recommend that the 
Conformance Clause link to the
terminology section of the spec


[PC issue]

"Principle" seems like an odd term to use to express what is basically a 
"Requirement". Here's a dictionary definition:
"[a] basic generalization that is accepted as true and that can be used 
as a basis for reasoning or conduct"
Suggest simply using the terms "Requirement" and "Good Practice".

These can be organized under "Principles"

[LH] This is more consistent with use of these terms in Web Architecture 
document
[general agreement] Note that the conformance clause will need to be 
revised accordingly

[LH] Quotes a "Principle" from the QA Handbook that contains the 
imperative voice
[PC] Imperative language contradicts the term "Principle"
[agreement] Switch back to the term "Guideline"


[KD issue] We need a better definition of "testability"

[all] We do use this term, and also "testable", "untestable"

[DH] Reads our current definition from the glossary

[PC] Suggests that we adopt this definition from Alex Rousskov:

"boolean condition X is testable" means that "there exist a finite
procedure to attain a high level of confidence that X is true".

(Note that what level of confidence is "high enough" is, essentially, up
to the tester to define for a given environment. The cost of the testing
procedure and the cost of a false answer, among other factors, would
affect that environment-specific definition.

No need to include these qualifications in the definition)


[Discussion of conformance clause template]

[AI] LH to review conformance clause template to ensure that it's in
sync with SpecGL. Also, ensure that SpecGL references to the conformance
clause template are informative rather than normative. Due 11/21.


[LR] Template should be in QAWG workspace. Is it OK if this isn't as
polished as SpecGL?

[agreement] Yes - this is OK, so long as we achieve synchronization by CR.

<break>


[DH issue - references section]

[DH] Jeremy Carroll has commented that we don't have a very extensive 
list of
references. Should we beef this up?
[KD] Not unless they are explicitly referenced in the spec.
[DH] Suggests that a more extensive list of informative references would 
be useful;
shows we are building on prior art.
[KD] Insists that if we don't directly reference them we shouldn't list them
[LR/PC] Suggest that we create a "Further Reading" section
[KD] That's OK

[AI] DH to email the mailing list asking for suggestions
[AI] DH to fix minor editorial issues


[DH issue - Extensions and error-handling]

[DH] Suggests that Good Practice 4.3D - "Define error handling for 
unknown extensions"
be incorporated into 4.3B - "If extensibility is allowed, define an 
extension mechanism"
and/or 4.5A - "Define an error handling mechanism".

[others] Discussion

[AT] Agrees with DH, but...
[DH] Withdraws the suggestion


[LR] Suggests switching this with 4.3C so that it's adjacent to the 
primary 'extensions' item.


[DH issue - Some examples in Principles and Good Practices are not 
well-chosen]

[LR] We should examine all Techniques and Examples

[KD] We should define what we want for these sections - breaking a 
Technique into several
steps is OK, as is presenting several Techniques.

[KD] Suggests using a bullet for each Technique, and if required, 
subsidiary numbered items
for steps.

* Technique A
  1. Step 1
  2. Step 2
  3. Step 3
* Technique B
* Technique C
  1. Step 1
  2. Step 2

[PC/LR] Suggest working through these items as a group, but do this when 
we've finished
our other business.


[KD issue - Principle A1.1 - Include a Conformance Clause]

[KD] A technology can be defined by multiple docs/specs at different 
levels of maturity.
How can we define a Conformance Clause that applies to the whole set? 
(Eg, CSS3.)

[PC] In the Java world we create an "umbrella specification" (meta 
specifiction) that
covers all.
[MS] This is a more generic problem.
[LR] Every document must contain the clause or point to it.
[KD] RDF has multiple docs, but they are at least moved forward together
[PC] If there are multiple docs/specs, there must be a high-level document
that pulls them all together.
[LR/PC] What about "component" specs that are referenced/included in 
multiple other specs, such
as XPath or Xinclude?
[PC] They must define conformance requirements that will be used 
by/incorporated into the
higher-level specs.
[MS] This discussion should be "higher level" (not buried in a single 
guideline)
[all] Where could we put such a discussion?
[PC] Suggests adding a Concepts or Terminology section
[MS] Beware trying to set W3C policy as a whole
[PC] We can make some recommendations: if you're creating a spec 
designed to be
incorporated into other specs, define your conformance clause 
accordingly, and if you're
creating a family of specs, create a higher-level 'umbrella spec' that 
defines the
relationship between the various components, and what it means to 
conform to the
collection as a whole
[AT] Remember that users/implementors will ultimately choose what pieces 
or components
they want to conform to.
[PC] WSI is an example of a "collective" or "integration" spec.
[PC] There are three models: standalone spec, spec intended to be part 
of a "family",
and spec intended to be incorporated into another spec.
[DH] A specification by definition includes normative content?

[agreement] Add a concepts section in the Introduction, addressing these 
issues.

[AI] DH will draft this section. Due 11/12.
[AI] KD will create graphics to illustrate. Due 11/12.
[AI] KD will raise this issue with the Advisory Board after our publication.


<adjourn>

Received on Thursday, 28 October 2004 16:15:03 UTC