Re: Testable assertion tagging for W3C specifications

Lofton,

comments inlined

On Friday, May 24, 2002, at 07:15 PM, Lofton Henderson wrote:

> Dimitris,
>
> There is a lot in this thread, and I think at the least that QAWG needs 
> to "mine" the thread for specific issues (points of contention from 
> past messages) that we should state as SpecGuide issues, and then 
> resolve.
>
> I also believe that some of the goals that Scott originally stated are 
> immediately realizable, and possibly without too much added burden on 
> spec editors.  The obvious realization is in xmlspec, but we should 
> also consider a less ambitious class-based markup scheme using XHTML 
> (which if properly designed, could automatically be transformed [e.g., 
> XSLT] later to xmlspec).

[dd] Right; we could either go for xmlspec, which I think would raise 
more protests that going for a class-based XHTML scheme. I personally 
feel quite good with xmlspec, but we'd have to do the survey I propose 
below to find out if that goes for enough people.

>  
>
> To recap, I include Scott's goals with your replies, then your "Road 
> ahead" proposal.  First the goals...
>
> At 10:48 PM 5/10/02 +0200, Dimitris Dimitriadis wrote:
>
> [...]
> General outline (in reply to Scott's original posting, parts of which 
> are pasted, my comments prefixed by [dd]):
>
> ---
> GOALS:
>    Allow an external document (test case, erratum, email, etc.) to point
>    directly at a "testable" normative sentence in a Recommendation.
>
> [dd] This would clearly simplify the task of, if we look at tests, 
> knowing which part in particualr is being tested, but requires 
> structure and issues tracking. This in turn implies that it may need to 
> be an intra-W3C "standard".
>
[dd] The "standard" I mention here would be the testable assertion 
markup we've discussed and the linking technique (pointing to a testable 
assertion from a part of the actual test, say).

>    Encourage document editors to view some of the sentences as "test
>    assertions" and to write them in a style that conveys precisely what
>    they declare.
>
> [dd] Any disambiguation is welcome. However, there are issues with, as 
> I see it:
>
> 1. Forcing specification authors to write in a particular markup (which 
> could be worth it, given the complexity they need to bea ble to master 
> to begin with to author a spec)
> 2. Different WG's will certainly have their own views on how the 
> xmlspe, for example, should be extended.
>
>    Explore possibilities for machine processing of testable sentences in
>    the future.
>
> [dd] This is already being done in an embryo-stage way, more on how we 
> generate the DOM Test Suite markup language below. We have had positive 
> responses from this approach, as it treats the specification (the XML 
> version, actually) more normatively than just serving to produce the 
> human readable HTML output.
>
>    Link error assertions to error catalogues (see the work that Mike 
> Kay is
>    doing with the XSLT document: (
>   http://www.w3.org/TR/xslt20/#error-summary)).
>    Provide a tagging scheme for testing of grammatical statements, such 
> as
>    the ad-hoc one employed in the XPath/XQuery specifications.
>    Possibly provide markup also for discretionary behavior.
>
> [dd] In general, I think the cost/benefit analysis woul end up with the 
> follwing (The QA activity, of course, tries to emphasize the benefits, 
> and for this and other reasons it is important that people out there 
> raise their voice at an early stage)
> [...snip details...]
>
>
> And now to your proposed way forward...
>
> Road ahead:
>
> 1. We should find out how many use xmlspec, or anything like it, now. 
> Here group chairs could help greatly.
> 2. We should play around with the idea of creating an extension to 
> xmlspec (or write it from the beginning) to represent semantics and 
> intended behaviours (again, please include me in the loop)
> 3. Any interested party should read the QA Specification Guidelines 
> document, comments are very much appreciated.
> 4. The topic of whether the HTML version found on the web or the 
> (generated) XML  version is the normative one should perhaps be made 
> again, as following the idea that originating this thread will 
> certainly mean that the XML version takes precedence over any other 
> version (when it comes to specification interpretation)
> 5. Find a relevant forum for discussing this. the QA IG list has been 
> proposed, and I think it may be the right place for it, at least for 
> now.
>
>
> It looks sensible to me.  I would add an item, to try to untangle the 
> numerous intermingled issues in this thread and enumerate a set of 
> discrete issues (for the QAWG Issues List).
>
[dd] Seems to be the proper way forward. I can start working on such an 
untaglement right away.

> Are you proposing to launch such a project, i.e., to initiate the first 
> steps? 
>
[dd] Yes, I do. I can build on this email you just sent, look through 
the thread again, and raise the separate issues we have at hand.

Scott, have you had time to look through the thread recently? Your views 
would be appreciated.

/Dimitris

> -Lofton.