Re: some questions about version information in the test suite

I'm getting to the point where uncertainty here is starting to hold up 

Could I suggest the following as a way of moving forward:

Assume that we are testing a processor that claims to implement XSD 
version P; if we want to test multiple processors or a processor that 
implements multiple schema versions, repeat the steps below with 
multiple values of P

(a) The version attribute can appear on testSuite (replacing 
schemaVersion), testSet, testGroup, schemaTest, or instanceTest. In each 
case the semantics are "if the attribute is present and includes the 
value P, then run this test / these tests; if it is present but does not 
include P, then skip this test. If the version attribute is absent, then 
run the test".

(b) The version attribute can appear on <expected>, meaning "these are 
the expected results for a processor at a given version". The test 
driver should find the <expected> element that either has no version 
attribute or whose version attribute includes P. There must be exactly 
one such element.

(c) Change the value of <expected> to be a list of permitted outcomes, 
where any of the listed outcomes is considered to pass the test. For example

<expected version="1.0">invalid notKnown</expected>
<expected version="1.1">invalid</expected>

indicates that acceptable results for 1.0 are "invalid" and "notKnown", 
whereas for 1.1 the only acceptable result is "invalid" This replaces 
the impDe attribute.

As a matter of test suite design, the aim should be to put the version 
attribute on the smallest element possible. Where XSD 1.0 and XSD 1.1 
exhibit interestingly different behaviors, there should be expected 
results for both. However, where the test uses XSD 1.1 syntax that will 
be rejected out of hand by an XSD 1.0, processor, there is little point 
in making XSD 1.0 processors run the test; it should simply be marked as 
version="1.1" typically at the testGroup or testSet level, rather than 
giving expected results for 1.0.

Michael Kay

Received on Monday, 21 June 2010 21:17:23 UTC