RE: tests/features updates

The XLSX cells that display as all hashes (####) are some kind of 
formatting thing when the text overflows the cell, at least that's what we 
surmised on earlier calls.
If you select one, the formula bar does show the text content, as does the 
exported XML that we XSLT into the HTML reports.
Our MS colleagues will have to speak to that.

Best Regards, John

Street address: 2455 South Road, P328 Poughkeepsie, NY USA 12601
Voice: 1+845-435-9470      Fax: 1+845-432-9787



From:
"Smith, Virginia (HP Software)" <virginia.smith@hp.com>
To:
"public-sml@w3.org" <public-sml@w3.org>
Date:
01/22/2009 12:25 PM
Subject:
RE: tests/features updates
Sent by:
public-sml-request@w3.org



My (3) comments inline below.
 
Also, can we fix the ####? 
 
--
ginny
 
From: public-sml-request@w3.org [mailto:public-sml-request@w3.org] On 
Behalf Of John Arwe
Sent: Saturday, January 17, 2009 11:03 AM
To: public-sml@w3.org
Subject: tests/features updates
 

Please read and respond via to "changes still needed" at a minimum, do not 
wait for the next telecon.  There is no point in dragging this out. 
Per last Thursday's discussion, enclosed are the updates we agreed I would 
make as well as a list of changes (further below) and a discussion of 
additional changes needed (next) that are NOT done in here yet because I 
considered them beyond what people unambiguously agreed to.  For those 
without Office 2007, I will attach PDFs of the features and tests sheets 
where the substantive changes were made. 
Kumar, you should use this as the base on which to add the Microsoft 
implementation's results.  Since I am not an editor, I cannot check them 
in myself.  Fortunately we have the email archive as a backup for my HDD. 
Changes still needed 
- Optional features 
We have discussed the need to straighten out the optional features issues 
several times now, e.g. how to specify that for a single test case the 
results may be different based on the feature set it tests intersected 
with the feature set present in the implementation.  The existing feature 
list tried to one feature into several, allowing each implementation to 
clearly say it supports the feature and allowing us to list "common" 
behaviors as required (e.g. when no schema binding elements are present, 
merge all schema definition documents).  The fundamental problem this 
fails to address is that a single test case may (since we paid some 
attention to this, in most cases will) have prescribed outputs (which may 
be different) both for implementations supporting the optional feature 
being exercised and for implementations not supporting the feature.  I 
think what we should be doing here is simply listing separate expected 
results, one for implementations that support the feature and one for 
implementations that do not.  For required features, the two results must 
be equal (and it is easy to have the XSLT building the impl report check 
that).  For optional features, they may differ depending upon what the 
spec says and how the test case itself is constructed.  For example, 
schema bindings: it is fairly straightforward to construct SMLIF documents 
where "schema binding aware" validators find a model valid, but 
"...unaware" validators find it invalid.  Assuming this works for people, 
we would: 
-- make the existing smlifBaseuri and xmlBase features both optional (now 
required) 
-- Have XSLT enforce the requirement for at least one of them in each 
implementation. 
<g>We should state that in the remarks for human readers</g>
-- remove the xxxNS features (rows 14 and 18)  <g> Looks like you did this 
already. What was the reason for this?</g>
-- (optional) change baseURI to relativeRef (seems closer to the 
description that one must establish a base URI if relative references 
exist in the model). 
-- update the export XSD (Kumar) 
-- update the XSLT to generate the impl report (me, easy once I have an 
xml sample to work from) 
- Schema binding test cases 
The COSMOS test plan generator excluded them (why, I have no idea), but a 
half dozen or so exist and they run just fine.  I'll get the relevant data 
next week and email the rows. 
- "Test cases to add" sheet 
I added the ones COSMOS offered us as rows 144-152 on the assumption that 
the working group's stance was still to use whatever COSMOS has until it 
is proven unusable (e.g. the duplicate sml:uri tests).  The working group 
needs to vet those and decide to keep them, toss them, etc.  See 
http://lists.w3.org/Archives/Public/public-sml/2009Jan/0013.html . I did 
update the -map formulae to cover these additions (and rounded up to 200). 

- Clarify derefNV feature 
Existing test cases exercise deref in the context of Schematron rules. The 
feature description leaves it unclear whether or not these test cases test 
the feature.  I know from looking at the COSMOS code that it would (it 
invokes XSLT to test the Schematron rules, so derefs in rules end up 
calling as an XPath function not something "native" inside the validator 
implementation).  If these test cases "count", row 26 (probably others) 
would need to be updated. 
- MS Results on locid tests 
NA seems inappropriate.  In this case, the expected results (since we 
define results purely in terms of SML/IF-validity) should be the same 
regardless of the implementations support or lack of it, if locid is the 
only optional feature involved.  SML I think does prescribe that locid 
must be ignored so it should not change validation results. <g>I don’t 
think it does.</g> 
Changes included in the files below 
- Corrected feature names on rows 53, 58, 74, 76-77, 96-99, 106-110, 116 
so they get counted on the feature map 
- 55, 56: examined test case, they have rule binding elements so I removed 
the ? on rule bindings and corrected the token so it gets counted. 
- 63-68: changed feature2 from smlref to baseUri ... in general on these I 
took the approach of making the primary feature == the particular base URI 
mechanism feature, and setting feature2 to baseUri. 
- added rows 144-152 (discussion above) 

Best Regards, John

Street address: 2455 South Road, P328 Poughkeepsie, NY USA 12601
Voice: 1+845-435-9470      Fax: 1+845-432-9787