Re: [www-qa] After the Workshop, still plenty of options to consider

Yes David, I think your taxonomy is a good starting point that will
allow us to evaluate what various WG are doing.

We need to track that since we can "impose" any particular level of QA
practice to the WGs without the knowlegde of what is going on in the
majority of groups.

I'd personally favor being firm about what the WGs do to improve their
specs (e.g. idenfication of assertions, normative section, clarity, no
ambiguity, etc) and what they do to coordinate with QA development,
whether it happens within or outside the group.

It'll be in the scope of the QA WG to establish this kind of WG/QA
metrics (a sort of companion table to TheMatrix)

> At the Workshop, I made some remarks about "floor expectations" for the
> Working Groups (WGs) in the future. We talked about many levels at which
> the floor could be set, but didn't aim for closure. Most likely, we'll
> have a rather low floor overall, then some WGs may set their goals higher.
> (This analysis is directed at WGs that are working on a substantive
> specification that is on track to become a Recommendation.) For your
> edification, here are all the levels I heard about, arranged in
> more-or-less increasing order of rigor or workload:
> 
> 0. Past practice: no firm requirement to address quality.
> 1. WG charter required to say something about QA plans, but could say
>    any of the following.
> 2. WG promises to think about testability when writing their documents.
> 3. WG makes an effort to have QA expertise among their membership.
> 4. Beyond a few normative examples, WG aims to have numerous normative
>    cases in the body of the Rec.
> 5. WG requires that a test suite, not necessarily complete and thorough,
>    exist somewhere before they go to Rec.
> 6. WG requires that they review and fully approve a test suite, not
>    necessarily complete and thorough, before they go to Rec.
> 7. WG commits to reviewing test cases on a continuing basis, including
>    after going to Rec.
> 8. WG will attempt to generate a complete catalog of test cases before
>    going to Rec.
> 9. WG insists on a complete catalog and partial implementation of a test
>    suite before going to Rec.
> 10. WG insists on a test suite, not necessarily complete and thorough,
>     and provides an official test harness, before going to Rec.
> 11. WG commits to writing test cases themselves and delivers a test
>     harness and suite at some point.
> 12. WG insists on a complete test suite before going to Rec, and
>     empowers W3C Team or endorsed Lab to operate a testing service.
> 
> Obviously, several kinds of mix-and-match are possible from the above
> ideas. Approval of test cases is a WG responsibility, but other tasks
> could be done outside the WG, and test cases could have mixed origin.
> Is the above a good structure for discussing WG expectations?
> .................David Marston

Received on Wednesday, 11 April 2001 07:05:37 UTC