Re: Testable assertion tagging for W3C specifications

I completely agree with Scott's sentiments here, as well as with Charles' 
in the previous message of this thread.  Because testability of 
specifications should be a priority issue, and not a afterthought, the QAWG 
is taking up this aspect of the QA Framework now.

In mid-May, we expect to publish 4 WD members of the Framework family plus 
two checklists (see the overview table at [1]).  Amongst these working 
drafts will be the first public Working Draft (FPWD) of "Specification 
Guidelines" and its companion checklist.  You can see a preview of these 
(Editor's drafts) at [2], [3].

The topic of spec anatomy (grammars for authoring) is addressed in the 
Introduction, and in Guidelines 12 and 13.  Discretionary behaviors were 
mentioned in this thread, and we address those in Guideline 8.  This is 
just the FPWD now, but we (QA) view this as the start of a major project -- 
both the development and progression of this specification, and work on 
actual techniques (specific markup).

One comment in the thread was interesting -- the startup hurdle of 
switching to xmlspec suggests the utility of a half-way step, such as some 
class definitions for XHTML authoring.

By the way, these techniques -- markup to enhance testablility, location of 
test assertions, and even direct generation of test suties -- are not 
speculative.  They're in use now.  An example that we have looked at in 
QAWG is DOM.

Regards,
Lofton.

[1] http://www.w3.org/QA/WG/#docs
[2] http://www.w3.org/QA/WG/2002/framework-20020506/qaframe-spec.html
[3] http://www.w3.org/QA/WG/2002/framework-20020506/qaframe-spec-checklist.html

At 03:45 PM 5/6/2002 -0400, scott_boag@us.ibm.com wrote:

> > This sounds like it might force a particular writing style and constrain
> > the sentence structures used by editors.
>
>Maybe.
>
> > I am concerned about anything that would increase the work load or the
> > constraints on editors.
>
>Personally I think that testability of a specification should be a priority
>issue.  Yes, it might increase work load in the short term.
>
>On the other hand, it might decrease workload in a way.  Too often we think
>of the specification as prose.  What it really must be is a series of
>testable statements.  Let someone else write a tutorial.  By thinking of
>specifications as a kind of legal document, you may reduce the overall
>amount of verbiage, and thus reduce the text and the number of errors that
>can creep in.
>
>But this is just some arguable speculation.  Really, hell if I know.  I do
>know that it's always a good idea to engineer with testing in mind.  Too
>often testing and verification is left as an afterthought.
>
>-scott
>
>
>
>
>|---------+-------------------------------------------->
>|         |           Jonathan Robie                   |
>|         |           <jonathan.robie@datadirect-techno|
>|         |           logies.com>                      |
>|         |                                            |
>|         |           05/06/2002 03:10 PM              |
>|         |                                            |
>|---------+-------------------------------------------->
> 
>  >----------------------------------------------------------------------------------------------|
>   | 
>                         |
>   |       To:       scott_boag@us.ibm.com, spec-prod@w3.org, 
> w3c-query-editors@w3.org            |
>   |       cc:       www-qa@w3.org 
>                         |
>   |       Subject:  Re: Testable assertion tagging for W3C 
> specifications                        |
> 
>  >----------------------------------------------------------------------------------------------|
>
>
>
>
>At 01:29 PM 5/6/2002 -0400, scott_boag@us.ibm.com wrote:
>
> >    Encourage document editors to view some of the sentences as "test
> >    assertions" and to write them in a style that conveys precisely what
> >    they declare.
> >    Explore possibilities for machine processing of testable sentences in
> >    the future.
>
>This sounds like it might force a particular writing style and constrain
>the sentence structures used by editors. Am I reading too much into this?
>
>I am concerned about anything that would increase the work load or the
>constraints on editors. We've got an enormous amount of work to do as it
>is.
>
>Jonathan