tests/features updates

Please read and respond via to "changes still needed" at a minimum, do not 
wait for the next telecon.  There is no point in dragging this out.
Per last Thursday's discussion, enclosed are the updates we agreed I would 
make as well as a list of changes (further below) and a discussion of 
additional changes needed (next) that are NOT done in here yet because I 
considered them beyond what people unambiguously agreed to.  For those 
without Office 2007, I will attach PDFs of the features and tests sheets 
where the substantive changes were made.
Kumar, you should use this as the base on which to add the Microsoft 
implementation's results.  Since I am not an editor, I cannot check them 
in myself.  Fortunately we have the email archive as a backup for my HDD.

Changes still needed
- Optional features
We have discussed the need to straighten out the optional features issues 
several times now, e.g. how to specify that for a single test case the 
results may be different based on the feature set it tests intersected 
with the feature set present in the implementation.  The existing feature 
list tried to one feature into several, allowing each implementation to 
clearly say it supports the feature and allowing us to list "common" 
behaviors as required (e.g. when no schema binding elements are present, 
merge all schema definition documents).  The fundamental problem this 
fails to address is that a single test case may (since we paid some 
attention to this, in most cases will) have prescribed outputs (which may 
be different) both for implementations supporting the optional feature 
being exercised and for implementations not supporting the feature.  I 
think what we should be doing here is simply listing separate expected 
results, one for implementations that support the feature and one for 
implementations that do not.  For required features, the two results must 
be equal (and it is easy to have the XSLT building the impl report check 
that).  For optional features, they may differ depending upon what the 
spec says and how the test case itself is constructed.  For example, 
schema bindings: it is fairly straightforward to construct SMLIF documents 
where "schema binding aware" validators find a model valid, but 
"...unaware" validators find it invalid.  Assuming this works for people, 
we would:
-- make the existing smlifBaseuri and xmlBase features both optional (now 
required)
-- Have XSLT enforce the requirement for at least one of them in each 
implementation.
-- remove the xxxNS features (rows 14 and 18)
-- (optional) change baseURI to relativeRef (seems closer to the 
description that one must establish a base URI if relative references 
exist in the model).
-- update the export XSD (Kumar)
-- update the XSLT to generate the impl report (me, easy once I have an 
xml sample to work from)

- Schema binding test cases
The COSMOS test plan generator excluded them (why, I have no idea), but a 
half dozen or so exist and they run just fine.  I'll get the relevant data 
next week and email the rows.

- "Test cases to add" sheet 
I added the ones COSMOS offered us as rows 144-152 on the assumption that 
the working group's stance was still to use whatever COSMOS has until it 
is proven unusable (e.g. the duplicate sml:uri tests).  The working group 
needs to vet those and decide to keep them, toss them, etc.  See 
http://lists.w3.org/Archives/Public/public-sml/2009Jan/0013.html . I did 
update the -map formulae to cover these additions (and rounded up to 200).

- Clarify derefNV feature
Existing test cases exercise deref in the context of Schematron rules. The 
feature description leaves it unclear whether or not these test cases test 
the feature.  I know from looking at the COSMOS code that it would (it 
invokes XSLT to test the Schematron rules, so derefs in rules end up 
calling as an XPath function not something "native" inside the validator 
implementation).  If these test cases "count", row 26 (probably others) 
would need to be updated.

- MS Results on locid tests
NA seems inappropriate.  In this case, the expected results (since we 
define results purely in terms of SML/IF-validity) should be the same 
regardless of the implementations support or lack of it, if locid is the 
only optional feature involved.  SML I think does prescribe that locid 
must be ignored so it should not change validation results.

Changes included in the files below
- Corrected feature names on rows 53, 58, 74, 76-77, 96-99, 106-110, 116 
so they get counted on the feature map
- 55, 56: examined test case, they have rule binding elements so I removed 
the ? on rule bindings and corrected the token so it gets counted.
- 63-68: changed feature2 from smlref to baseUri ... in general on these I 
took the approach of making the primary feature == the particular base URI 
mechanism feature, and setting feature2 to baseUri.
- added rows 144-152 (discussion above)



Best Regards, John

Street address: 2455 South Road, P328 Poughkeepsie, NY USA 12601
Voice: 1+845-435-9470      Fax: 1+845-432-9787