- From: Charles McCathieNevile <charles@w3.org>
- Date: Mon, 6 May 2002 17:01:39 -0400 (EDT)
- To: Alex Rousskov <rousskov@measurement-factory.com>
- cc: Jonathan Robie <jonathan.robie@datadirect-technologies.com>, <scott_boag@us.ibm.com>, <spec-prod@w3.org>, <w3c-query-editors@w3.org>, <www-qa@w3.org>
On Mon, 6 May 2002, Alex Rousskov wrote: There are many ways to "enable our specifications to be tested". And we should not forget that ability to test is not the only goal. Absolutely. But not being able to test whether a specification is being met means that it is less a specification than a general description of an idea. An ideal specifications would be both readable by humans and testable by machines. Most current specifications, especially complex and interesting ones, do not satisfy both of these properties. There can be two general directions for improving the situation: - making humans write machine-testable text (code) - making machines understand human-oriented text Some steps can be done in both directions. For example, making every bit of a spec addressable is not much extra work for humans and helps machines a lot. EARL takes this approach - it simply requires that there is an identifiable test - i.e. it can be addressed, and the test itself is assumed to be specified in human-readable language. For some categories of test it is possible to compare a reference example to a test example (e.g. visually), and for others it is useful to provide the relevant part of the specification (which is assumed to explain what the test is) along with the example being tested (e.g. the CSS test suite, or WAI checkpoints). There is another class of cases where it is possible to do the testing - e.g. validation of various kinds, but where it is difficult to build general tools that can use the results of testing because there is little machine-readable information to identify which requirement was failed. I agree that a useful step would be to make each requirement addressable, and to work with that before we go further into this realtively uncharted territory. (Well, there are organisations who have spent years on this. Maybe some of them are represented on this list). cheers Chaals Interestingly, having smart enough machines is sufficient for humans to write testable specs because a smart machine would tell the author whether what s/he just wrote is testable. Compare this with writing poems using pre-declared Java identifiers to avoid spelling mistakes and having a spelling checker that highlights misspelled words as you type your poem. Personally, I do not like the idea of making humans write more code than they already have to. I think it is a counter-productive approach in the long term, especially since not all real specifications can have 100% test coverage (some requirements cannot be pragmatically tested at all). Making spec bits addressable is as far as I would go [today and within W3C scope]. My understanding is that we already have the tools to make such addressing. And, practically speaking, it is not too much work to apply any given addressing scheme to address every interesting piece of a given specification. Do we really need a one-for-all standard that editors would be mandated to use? Thanks, Alex. -- Charles McCathieNevile http://www.w3.org/People/Charles phone: +61 409 134 136 W3C Web Accessibility Initiative http://www.w3.org/WAI fax: +33 4 92 38 78 22 Location: 21 Mitchell street FOOTSCRAY Vic 3011, Australia (or W3C INRIA, Route des Lucioles, BP 93, 06902 Sophia Antipolis Cedex, France)
Received on Monday, 6 May 2002 17:01:43 UTC