Review of the WCAG WG versus QA Operational Guidelines

Dear WCAG working group,

Here is, with some unexpected delay (in a word - flu - sorry about that)
a review of the WCAG Working Group, based mostly on the analysis of
two documents (your new charter and your Implementation Testing Notes
document) against the QA Operational Guidelines and some discussion with
Wendy on these.

Obviously the WCAG WG, or at least part of it, is already familiar with
QA, and it is excellent that you have already had a look at the Ops GL
(and Test GL).

I was amazed to see a QA commitment referring to the QA Framework in
your proposed charter, congratulations. You apparently used an older WD
of the QA operational guidelines as your basis for reflexion, which
should not be a real problem, but as I told Wendy, I do encourage you 
to have a look at the more recent, and hopefully clearer, 
http://www.w3.org/TR/qaframe-ops/
which has a different - easier - approach on QA commitment.

You chose to commit to QA level 3 according to the feb WD of QA Ops.
That's a reasonable choice, though (still based on this version of QA
Ops) level 5, with the addition of test assertions for the specification
and a test case contribution/review process seems to be closer to what
the charter appears to aim at.

[the above makes you pass guideline 2 of QA-Ops, BTW]

Thanks to clarification by Wendy on a preliminary review of your group I
know that your call for participation may try to bring in, in addition
to the current participants, a new perspective on your work.

To that effect I would highlight the second guideline of QA-Ops and
the fact that it suggests that the CFP includes a specific request
for inclusion of QA experts into the WG from participating members.
Apparently some of the group members are already involved in Conformance
testing, but a clear mention of QA needs in the CFP could bring you new
people with a fresh vision and a focus on Testing.

[the above would make you pass guideline 2 of QA-Ops, BTW]

I'm moving on to Gl4 of QA-ops, and now focusing also on your test
document.

I note that the inclusion of the QA moderator into the charter is great.
Truly impressing. I'm looking forward to the nomination of the QA
moderator for the WCAG group (and thank Wendy a lot for acting as point
of contact in the meantime).

Your Implementation Testing document is a very good draft of what we
call a QA process document. By QA process document we're actually
talking about a document clarifying " quality-related logistical and
communications setup, contribution and licensing policies, publication
and maintenance plans, and other critical process and operational
details". And as you may expect, a large part of this is defining, by
whom, when, where and how will tests be processed.

Your approach of testing is already quite mature, since your charter
implies that you will have a review of test materials (which happens to
be a checkpoint of QA-Ops). One thing that seems to be missing, though,
(correct me if I'm wrong) is a sense of planning/milestones/versioning
for your test work. There again, the most recent QA-Ops-GL should help.

I think the next step in our WCAG/QA cooperation may be to think a bit
further about this document. The process of working on such a document
should help you address all the details about testing (and quality),
even those you may not have thought of yet, such as:
- planning the development of your Test framework
- addressing test material publication, including a contribution process
- licensing terms for the test material you are planning to produce, 
- etc.

More info: 
http://www.w3.org/TR/qaframe-ops/qaframe-ops#Ck-produce-process-doc
(and more generally, guidelines 4 to 8 of QA-Ops)

As your contact in the QA activity I will, during this phase of work,
try to guide you in your interpretation and use of the QA Ops Guidelines
when developing this "QA Process / Test plan".

I think it is quite safe to say that when this document is completed,
the QAWG working group will certainly (modulo maybe minor details) be
"QA Ops AA compliant", if not AAA. That is, for both the WCAG and QA
WGs, excellent news.

******

Comments, questions and clarifications on this review are absolutely
welcome.

I am also looking forward to the next publication of WCAG 2.0, which 
as promised I will review against the latest QA Specification 
Guidelines.

thank you.
-- 
olivier

Received on Tuesday, 9 December 2003 02:06:30 UTC