Review of QAH

This is a personal review of the QA Handbook WD dated 10 May 2004.


Overall I am happy with this WD and believe that this represents
a significant forward step over the previous round of QAF publications.
I offer the QA WG my congratulations on having turned things around
so quickly.

Only three of these comments are not intended as formal comments 
according to the W3C process, these are marked with a *

Status of this Document

* Links to latest version of QA Introduction and QA Ops Guidelines
(sorry I have not checked this, unfortunately I am offline as I
am writing this up, and cannot check immediately).
I suggest the latest version links of QA Intro and QA Ops should
either point to QAH or to a dummy page indicating that they have
been superseded by QAH. If this is already the case, sorry for
having raised this in error.

1.3 "QAH takes this approach for several reasons
....
... too authoritarian and fierce"

I believe this rationale is interesting only to those
relatively few readers who have reviewed the older QAF
and wish to understand the change. Hence, while it was
very much in place in this WD, I suggest it should be
deleted in the next WD.

1.3 "If it looks helpful, follow it. If not, don't
-- there's no consequence"

Suggest delete "there's no consequence" - you don't really
believe it, and, FWIW, neither do it. Choosing to not follow
a suggestion is likely to result in lower quality,
but that decision may be the right one to make in particular
circumstances. This observation could be made directly in
the text. e.g.
"These suggestions will generally be helpful and
enhance the quality of the work of a WG. However,
each suggestion should be applied or not depending
on the specific situation of the WG."

1.4 QAH Scope, last para

In my view it would be better to be more upfront about
the degree of emphasis on test and testability in the QAF,
and note that WG's will need to balance this emphasis
with other quality issues which are either only partially
addressed or not addressed at all in the QAF, e.g. timeliness.
(related to * comment on 6.1)


2. Story at beginnning

I like it.

2. "Different kinds of QA deliverables might include"

Suggest also

"+ tests of new, changed or contentious functionality"

this would capture the test driven spec development, that I presented in
Cannes.

2 Note: I have not yet reviewed the draft charter template
(I now doubt I will be able to do so before 12 Jun)

2. How can I do it? second bullet point
suggest rephrase, I particularly disliked the wording
"now is a good time for QAH to acknowledge that "

Perhaps
s/Having adovcated/While often/
s/now is a good time for QAH to acknowledge that//

the current wording seems somewhat battle-scarred to me.

3. Day-to-day QA operations

first story

I think TM is being used without being defined here.

fourth para: "The W3C Process Document ..."
suggest delete "all important"

* 3.1 QA Moderator para
suggest
s/a single identified/an identified/

e.g. WebOnt had two people filling this role without problem.

3.1 Note: I have not reviewed QAPD template, again I fear that I am
out of time for this round

4. on the Document License
   I wondered about copy&paste issues.
   At least one function of a test suite is as a set of examples
which should work. Hence it is tempting to do a view source and
copy and paste from them. I note that this might be something
that should be discouraged, particularly when the tests are
exercising corners of the spec rather than the main body
No suggested change, however, I wonder if you might want to add
something on this.

5. Acquiring test materials

Hmm, on my print out I have scribbled "links to test" alongside the
first "good practice" box and the following two paras. However, I
can't think what I was on about :(

Maybe that the two lists in the two paras following the "good practice"
box should consist of items each hyperlinked into a relevant
part of the test guidelines where the issue is expanded.

5. TM

The abbreviation TM is expanded here as Test Materials, but it
has already been used extensively


6.1 WG spec editors and authors

typo: missing "of" in "understanding the Principles"

* 6.1 WG-TS Moderator
versus "WG's QA projects"

Remaking one of my comments on the
CR version of QAF, that it is really a test and testability
framework and the name QA framework is unhelpfully broad.
My prefered solution is to rename the QAF to be a
"Test and Testability Framework".
I suspect that the QAWG is unwilling to concede this.
This issue is partially addressed in section 1.4 scope,
and as I have already suggested could be made clearer there.
In this section suggest
s/WG's QA projects/WG's TM/

Also not clear what TS in WG-TS participant or WG-TS moderator is
meant to stand for. Maybe s/TS/TM/

6.1 Last words
I am out of time on checking that QAIG has chartered deliverables.
I thought in general IG's did not have deliverables, and so the
last sentence jarred.

6.2 Primer

In the current WD this primer felt redundant, because the WD
itself was sufficiently short. However, I suspect that when all
the TBDs are done, the QAH will maybe double in length (hopefully
not quite); and then the primer will become more significant.
I suggest that while the main copy is part of the QAH single html
file, that it may well be useful for many readers to have the
primer as a short standalone file.

I note that having reviewed your documents on a number of
occassions now, I am not the right person to tell whether the
primer was redundant or not.

8. Refs

QAF-TEST

I don't believe this doc has made it to Candidate Rec as suggested
in the note. Anyway the note will go in the next revision (after
the next version of QAF-TEST has been published).


===



Jeremy Carroll



PS the WebOntWG acceptance of your disposition of comments
was the very last action of that WG, which has now disbanded.

Received on Thursday, 3 June 2004 09:54:02 UTC