My action item AI-20020614-05 completed [Testable assertions thread recap]

Action item AI-20020614-05: Send e-mail to wg with the list of items 
that came up from the thread on testable assertions

This email consists of the body of a message sent to the QA IG list.

All,

I've an action item to send to the WG list the discussion that took 
place on the IG list on testable assertions a while back (May 6 - June 
30, 2002) in issues form. The thread was started by Scott Boag, IBM in 
this email: 
http://lists.w3.org/Archives/Public/www-qa/2002May/0000.html.

This email is FYI; I'll try to start the thread again as it has died. I 
should have more substantive issues to bring up now that we've 
discussed this in the WG. Your feedback is important.

Below you'll find a list of items with their respective originator and 
the people that discussed it.

The original proposal is to add structure to XML Spec, allowing for 
specifications to contain "testable assertions" directly in the 
specification itself (in order to allow for external documents to point 
to the TA, encourage document editors to view some of the statements as 
TA and to allow for writing conveying precisely that, explore 
possibilities for machine processing of such assertions. Very close to 
my own thoughts on the matter as written in both Spec and Test GL, as 
per emails on this topic to this list in the past. XPath/XQuery/XSLT 
proposed as candidates. Originator: Scott Boag

Argument against this: very few specs are written using XML Spec (or 
XHTML with classes). The survey reported on earlier proves this wrong 
(as I anticipated when I proposed we go ahead with it).

Imposing too many restrictions on editors may add to their workload, 
which is big enough already (Jonathan Robie)

May be the case, but improves quality of specifications (Charles 
McCathieNevile)

Is there a need for a one-for-all standard which editors would be 
mandated to use? 100% testability is not possible. A specification is 
primarily meant to enable building compliant applications, not to 
enable conformance testing. (Alex Rousskov)

XHTML+classes can be an intermediate step toward XML Spec (Karl Dubost)

Need to investigate specification authoring techniques, to see if 
appropriate tools for representing testable assertions are being used 
(Dimitris Dimitriadis)

Pointing to specifications need not be don in one single way. Diversity 
has its merits (Alex Rousskov)

Need to gather requirements (which is what this posting aims at 
starting)

Lots and lots of discussion about whether we inline explicit assertions 
or derive them from the specification (implicit information)

Discussion about whether a good enough pointing mechanism (to identify 
contents of a specification and test assertions therein) is enough for 
our needs (Alex Rousskov et al)

Should we control the specification markup alone? (D. Marston, D. 
Dimitriadis) Or have a fair markup and a fair adressing scheme? (A. 
Rousskov, S. Boag)? Perfect markup would solve our problems, but may be 
difficult to bring about (however, conclusions from the specification 
authoring survey render a uniform markup idea fairly plausible)

Discussion on whether a better markup should be mandatory, or a tool

Cost/benefit analysis needs to be made

There was more technical and detailed discussion going on, but before 
we decide on a few first steips, I don't think we benefit from 
referring to that. I did in a post propose the following roadmap:

Road ahead:

1. We should find out how many use xmlspec, or anything like it, now. 
Here group chairs coudl help greatly.
2. We should play around with the idea of creating an extension to 
xmlspec (or write it from the beginning) to represent semantics and 
intended behaviours (again, please include me in the loop)
3. Any interested party should read the QA Specification Guidelines 
document, comments are very much appreciated.
4. The topic of whether the HTML version found on the web or the 
(generated) XML  version is the normative one should perhaps be made 
again, as following the idea that originating this thread will 
certainly mean that the XML version takes precedence over any other 
version (when it comes to specification interpretation)
5. Find a relevant forum for discussing this. the QA IG list has been 
proposed, and I think it may be the right place for it, at least for 
now.

1. We have undertaken a survey (which is 1. above). The results should 
be available soon, and then the discussion could start again.
2. Also, there was a proposal on an extension to XML Spec which, 
pending on resources, wasn't fleshed out.
3. We would still look forward to your comments on the QA WG documents 
produced so far (http://www.w3.org/QA)
4. Not adressed
5. QA IG list (plus possible chairs list) seems the appropriate forum.

Waiting for the results of the survey to be made public, that in turn 
will shed light on the cost/benefit aspect as well as how steep the 
learning curve is, and in addition the degree to which XML Spec 
contains all we need, I'd propose that we revisit the topic of trying 
to find:

1. The scope of this project (what it is we want to achieve)
2. Resource issues (I've proposed to the QA WG that a Test Task Force 
be formed within the QA WG, in order to deal with, among other things, 
testable assertions in specifications). Here all help is appreciated

Olivier, please update the action items list accordingly.

Best,

/Dimitris

Received on Friday, 17 January 2003 15:40:19 UTC