Re: Testable assertion tagging for W3C specifications

Dimitris,

There is a lot in this thread, and I think at the least that QAWG needs to 
"mine" the thread for specific issues (points of contention from past 
messages) that we should state as SpecGuide issues, and then resolve.

I also believe that some of the goals that Scott originally stated are 
immediately realizable, and possibly without too much added burden on spec 
editors.  The obvious realization is in xmlspec, but we should also 
consider a less ambitious class-based markup scheme using XHTML (which if 
properly designed, could automatically be transformed [e.g., XSLT] later to 
xmlspec).

To recap, I include Scott's goals with your replies, then your "Road ahead" 
proposal.  First the goals...

At 10:48 PM 5/10/02 +0200, Dimitris Dimitriadis wrote:
>[...]
>General outline (in reply to Scott's original posting, parts of which are 
>pasted, my comments prefixed by [dd]):
>
>---
>GOALS:
>    Allow an external document (test case, erratum, email, etc.) to point
>    directly at a "testable" normative sentence in a Recommendation.
>
>[dd] This would clearly simplify the task of, if we look at tests, knowing 
>which part in particualr is being tested, but requires structure and 
>issues tracking. This in turn implies that it may need to be an intra-W3C 
>"standard".
>
>    Encourage document editors to view some of the sentences as "test
>    assertions" and to write them in a style that conveys precisely what
>    they declare.
>
>[dd] Any disambiguation is welcome. However, there are issues with, as I 
>see it:
>
>1. Forcing specification authors to write in a particular markup (which 
>could be worth it, given the complexity they need to bea ble to master to 
>begin with to author a spec)
>2. Different WG's will certainly have their own views on how the xmlspe, 
>for example, should be extended.
>
>    Explore possibilities for machine processing of testable sentences in
>    the future.
>
>[dd] This is already being done in an embryo-stage way, more on how we 
>generate the DOM Test Suite markup language below. We have had positive 
>responses from this approach, as it treats the specification (the XML 
>version, actually) more normatively than just serving to produce the human 
>readable HTML output.
>
>    Link error assertions to error catalogues (see the work that Mike Kay is
>    doing with the XSLT document: (
> 
><http://www.w3.org/TR/xslt20/#error-summary>http://www.w3.org/TR/xslt20/#error-summary)). 
>
>    Provide a tagging scheme for testing of grammatical statements, such as
>    the ad-hoc one employed in the XPath/XQuery specifications.
>    Possibly provide markup also for discretionary behavior.
>
>[dd] In general, I think the cost/benefit analysis woul end up with the 
>follwing (The QA activity, of course, tries to emphasize the benefits, and 
>for this and other reasons it is important that people out there raise 
>their voice at an early stage)
>[...snip details...]

And now to your proposed way forward...

>Road ahead:
>
>1. We should find out how many use xmlspec, or anything like it, now. Here 
>group chairs could help greatly.
>2. We should play around with the idea of creating an extension to xmlspec 
>(or write it from the beginning) to represent semantics and intended 
>behaviours (again, please include me in the loop)
>3. Any interested party should read the QA Specification Guidelines 
>document, comments are very much appreciated.
>4. The topic of whether the HTML version found on the web or the 
>(generated) XML  version is the normative one should perhaps be made 
>again, as following the idea that originating this thread will certainly 
>mean that the XML version takes precedence over any other version (when it 
>comes to specification interpretation)
>5. Find a relevant forum for discussing this. the QA IG list has been 
>proposed, and I think it may be the right place for it, at least for now.

It looks sensible to me.  I would add an item, to try to untangle the 
numerous intermingled issues in this thread and enumerate a set of discrete 
issues (for the QAWG Issues List).

Are you proposing to launch such a project, i.e., to initiate the first 
steps?

-Lofton.