- From: Maciej Stachowiak <mjs@apple.com>
- Date: Fri, 8 Feb 2008 16:51:18 -0800
- To: Charles McCathieNevile <chaals@opera.com>
- Cc: Chris Wilson <Chris.Wilson@microsoft.com>, Sunava Dutta <sunavad@windows.microsoft.com>, "public-webapi@w3.org" <public-webapi@w3.org>, Gideon Cohn <gidco@windows.microsoft.com>, Zhenbin Xu <zhenbinx@windows.microsoft.com>, Marc Silbey <marcsil@windows.microsoft.com>, Ahmed Kamel <Ahmed.Kamel@microsoft.com>
On Feb 8, 2008, at 12:03 PM, Charles McCathieNevile wrote: > > On Fri, 08 Feb 2008 22:22:59 +0530, Chris Wilson <Chris.Wilson@microsoft.com > > wrote: > >> 2) In fact, on that note, we're interested to see the test suite be >> linked, normatively if necessary. > > Yes. I think this is a valuable piece of feedback. Currently W3C > process doesn't require test suites until you're trying to get out > of CR and I think it would be better to have them earlier. I agree that official test suites should be developed earlier than CR. Thorough test suites are critical to identifying implementation issues, and especially important for specs written after the fact for de facto standard technologies. This is a case where it is hard to tell if we have met our technical goals without a test suite. Fortunately, we have unofficial test suites as a starting point. However, I think that per standard practice the test suite should not be considered normative, only the text of the spec. In particular, conformance requirements that are not covered by a test must still be binding, and in case of conflict between the test suite and the spec, the spec must win. Of course, if the test suite and the spec ever disagree we will have to publish bug fixes to the test suite or errata to the spec, but in the meantime we need to be clear which is normative. Regards, Maciej
Received on Saturday, 9 February 2008 00:51:39 UTC