Re: QA WG review through OpsGL

Thanks, Dom, for getting this review done.  It is slightly embarrassing 
that QAWG fails OpsGL!  But maybe this will give us some impetus to finish 
our own process document (QAWGPD), which should take care of lots of the 
checkpoints, and address a couple of other conceptual issues (e.g., test 
materials).

There are OpsGL issues revealed.  QUESTION.  Do you want to queue these, 
using the LC form (see comments/questions about this at end)?

Some specific comments embedded...

At 04:56 PM 2/24/03 +0100, you wrote:
>Greetings,
>
>I've just completed my review of the QA WG through the OpsGL:
>http://www.w3.org/QA/Group/2003/02/qaframe-ops-qawg (linked from the
>review matrix:
>http://www.w3.org/QA/Group/2002/06/reviews )
>
>The summary of this review is:
>"Regarding how we as a WG pass the GL, we're currently in a very bad
>shape, mainly due to the lack of our QA process document, and also by
>lack of discussion on the topics recommended by the OpsGL: for instance,
>we don't have a clear plan to build a test suite for our documents.

This is related to an issue that I noticed when editing OpsGL.  Just what 
would qualify as "Test Materials" for OpsGL and for SpecGL, anyway?

Are the Ops-ICS and Spec-ICS test materials?

One could envision a validator for SpecGL (in theory -- lotsa' practical 
issues however).  What else would be a TM for SpecGL?

What else (other than maybe Ops-ICS) would be a test material for OpsGL?

Btw, I noticed it when I was editing OpsGL CP6.4 "Provide a conformance 
verification disclaimer with the test materials."  We pointed to OpsGL 
section 4.5, "Conformance disclaimer", as an example.  But it has nothing 
to do with test materials, so it is not an example of CP6.4.  (Therefore, I 
removed the example.)


>Regarding the OpsGL themselves, a clear issue that has been identified
>earlier is about the implicit timeline in the document: most guidelines
>are only applicable at some point in the WG's life, but the GL don't
>identify this aspect: this is something that absolutely needs to be
>stressed, and could even be used as a strategy to organize the GL as a
>whole, e.g. what you need to do before starting a WG, what needs to be
>done when you start developing a new spec, what needs to be done when
>you envision building a test suite, etc.

Issue.  (Are you going to enter it?)


>Less generally, I think the document would be improved with:
>
>     * a skeleton for a QA process document (probably as part part of the
>ExTechs document)

Are you thinking of something different than

http://www.w3.org/QA/WG/2003/02/OpsET-qapd.html ,

which is linked from all QAPD-related checkpoints of OpsET (13 of 'em)?

>     * cleaning the mandatoriness of the QA process document, i.e. a CP
>should still apply in most cases without a QA Process Document [I think
>I was one of those thinking it didn't matter because a Process Document
>is a priority 1 requirement, but thinking to it again, it makes more
>sense not add unnecessary dependencies].

I agree, in principle.  Documentation in the QAPD was a way to make the 
requirements testable.  I think it is a good way (and fairly easy, given 
the template.)  There could be other ways.  Alternative wording or 
proposal?  (Or ... LC-issue?)

>     * the table of GL 1 should probably be removed and reworked as
>suggested by Lynne recently
>"

When I worked closely on the table during OpsGL editing, I noticed a bunch 
of oddities and inconsistencies.  Lynne's issue/proposal is 
interesting.  (I'll log my own issues/observations about the table, just to 
be complete.)


>I'm unsure whether I need to open last call issues for the points I've
>identified during this review.

We will need to deal with them -- discuss and resolve -- one way or the 
other.  So we either tag the numerous points you made in the review and in 
this message (e.g., DHM-1, DHM-2, ...) and go through them, or enter them 
in the LC issue system.  Is there a downside to the latter?  (Or ... do you 
think that they are all straightforward, and the LC-issue system is too 
much machinery for them?).

Regards,
-Lofton.

Received on Monday, 24 February 2003 20:16:29 UTC