Re: early version of the XML Query Test Suite

On Thu, Aug 11, 2005 at 11:47:14PM -0700, Per Bothner wrote:
> I welcome and appreciate this test suit.  However, I'm missing a 
> rationale for the way it is laid out and how tests are written.

Per, and others...

I don't speak for the testing task force who made the test
suite here, but since they may mostly be on vacation, I'll take
the time to note that the test suite's goal is, strictly speaking,
to test whether the specification can be implemented.  It's not
a conformance test -- an implementation that passes all the tests
it's supposed to test might be conformant, but the test suite
doesn't prove it one way or the other.  On the other hand, if we
can compare the results from a group of implementations, we can
get an idea if there are features in the specification that are
not widely implemented, or where there might be interoperability
problems.  As at IETF (for example), a W3C spec must show at least
two interoperable implementations of each feature before it can
reasonably become a Recommendation.

So the test cases aren't written in the eXtreme or agile programming
sense of identifying bugs, but rather to give coverage of the
specification, and this different goal leads to a different approach.

I hope this helps -- of course, it's also possible that the published
test suite might be changed to make it easier to submit new entries,
as long as the new test cases are tied back to the specification.

best,

Liam

-- 
Liam Quin, W3C XML Activity Lead, http://www.w3.org/People/Quin/
http://www.holoweb.net/~liam/
Pictures from old books searched with XML Query:
http://fromoldbooks.org/Search/

Received on Saturday, 13 August 2005 00:14:06 UTC