Re: [www-qa] After the Workshop, still plenty of options to consider

David,

To your collection -- which I think is a valuable contribution to the WS 
output -- I have one addition and one question.

Addition.  "The WG does a rigorous analysis of the document, enumerating 
Test Requirements (Semantic Requirements), and possibly synthesizing a set 
of Test Purposes as well.  It might be someone else who actually builds or 
assembles the test cases."  This is a division of responsibility, WG and 
non-WG (or not necessarily WG), which has been discussed in the past.  I'm 
not sure exactly where it fits into the hierarchy.  It can be a fair amount 
of work (we have a couple of data points -- individual functional sections 
analyzed -- from the SVG WG).  Done properly, it is effectively a blueprint 
of what must be in the test suite(s).

Question.  About #4 below, I'm puzzled by "normative examples".  In my 
experience, it is common that examples are informative.  Else, you have a 
normative description and a normative example of the same functionality, 
and they could be inconsistent.  (Which has precedence if they 
disagree?).  For reference, in ISO specifications (and in the required ISO 
document styles), "EXAMPLE:" prefaces all examples and is explicitly 
connotes that what follows is non-normative (in fact, what follows is 
further offset by some difference in text style, such as smaller text size).

Regards,
-Lofton.


At 05:01 PM 4/6/01 -0400, David_Marston@lotus.com wrote:
>At the Workshop, I made some remarks about "floor expectations" for the
>Working Groups (WGs) in the future. We talked about many levels at which
>the floor could be set, but didn't aim for closure. Most likely, we'll
>have a rather low floor overall, then some WGs may set their goals higher.
>(This analysis is directed at WGs that are working on a substantive
>specification that is on track to become a Recommendation.) For your
>edification, here are all the levels I heard about, arranged in
>more-or-less increasing order of rigor or workload:
>
>0. Past practice: no firm requirement to address quality.
>1. WG charter required to say something about QA plans, but could say
>    any of the following.
>2. WG promises to think about testability when writing their documents.
>3. WG makes an effort to have QA expertise among their membership.
>4. Beyond a few normative examples, WG aims to have numerous normative
>    cases in the body of the Rec.
>5. WG requires that a test suite, not necessarily complete and thorough,
>    exist somewhere before they go to Rec.
>6. WG requires that they review and fully approve a test suite, not
>    necessarily complete and thorough, before they go to Rec.
>7. WG commits to reviewing test cases on a continuing basis, including
>    after going to Rec.
>8. WG will attempt to generate a complete catalog of test cases before
>    going to Rec.
>9. WG insists on a complete catalog and partial implementation of a test
>    suite before going to Rec.
>10. WG insists on a test suite, not necessarily complete and thorough,
>     and provides an official test harness, before going to Rec.
>11. WG commits to writing test cases themselves and delivers a test
>     harness and suite at some point.
>12. WG insists on a complete test suite before going to Rec, and
>     empowers W3C Team or endorsed Lab to operate a testing service.
>
>Obviously, several kinds of mix-and-match are possible from the above
>ideas. Approval of test cases is a WG responsibility, but other tasks
>could be done outside the WG, and test cases could have mixed origin.
>Is the above a good structure for discussing WG expectations?
>.................David Marston


*******************
Lofton Henderson
1919 Fourteenth St., #604
Boulder, CO   80302

Phone:  303-449-8728
Email:  lofton@rockynet.com
*******************

Received on Wednesday, 11 April 2001 15:16:20 UTC