Re: Action: Test Cases passing only 1 implementation

Hi Keith,

I am wondering if it would be possible to update this list.

The recently published EMC reports conformance for a number of tests that 
are listed below as supported only by FF, so this could reduce our number 
of "critical" and "high" priority features.

However, in that same report, there are some tests that EMC reports it 
does not conform to, but which are listed below as being supported by EMC 
only.

Finally, this makes me wonder if a review of the tests is needed to 
determine if there are any tests which have to go down to one 
implementation (FF).

In general, it seems more like we need the implementation report that 
looks like all your other reports, except that the last column would just 
contain a comma separated list of the implementations (FF, EMC, Chiba, 
Ubiquity, ...).  It seems like this could be generated by XSLT, and it 
would be the best way to get updates as the news changes.

Anyway, this caught my eye because I noticed that the digest() function 
was supported by EMC, but I remembered seeing red for those tests in their 
report.  Then I looked and noticed there were tests in Chapter 3 that EMC 
passes, but which are shown below as only supported in FF.  Then I found 
that all Ch. 4 tests listed below as supported only be EMC are in fact not 
supported by EMC.  That's when I stopped because it became clear that the 
automated approach would be needed.

Thanks,
John M. Boyer, Ph.D.
STSM, Interactive Documents and Web 2.0 Applications
Chair, W3C Forms Working Group
Workplace, Portal and Collaboration Software
IBM Victoria Software Lab
E-Mail: boyerj@ca.ibm.com 

Blog: http://www.ibm.com/developerworks/blogs/page/JohnBoyer
Blog RSS feed: 
http://www.ibm.com/developerworks/blogs/rss/JohnBoyer?flavor=rssdw





From:
Keith Wells <wellsk@us.ibm.com>
To:
Forms WG <public-forms@w3.org>
Date:
01/19/2009 11:33 AM
Subject:
Action: Test Cases passing only 1 implementation



Summary for Tests Passing in only 1 implementation: Firefox and EMC XForms 
1.1 implementation reports

Action: Analyze the Firefox and EMC implementation reports to find the 
test cases which have only one successful pass.
Note: Even though the Ubiquity-XForms project [1] has not officially 
submitted a formal implementation report for XForms 1.1, these reports are 
available and some test cases with successful passes were used in this 
summary.

Note: MUST/Required, SHOULD/Recommendation and MAY/Optional were derived 
from [2]. 

The following represent the tests which passed in only one implementation.

>From chapter 3:
Required:
3.3.1.c1 valid schema (EMC)
3.3.1.c2 invalid schema (EMC)
3.3.1.d2 version attribute negative test 1 (FF)
3.3.1.d3 version attribute negative test 2 (FF) 
3.3.4.b more bind element examples (FF)

>From chapter 4:
Required:
4.2.1.b2 schemas loaded with an xforms-link-exception (EMC)
4.3.6.b navigation sequence with navindex (FF)
4.6.1.a2 event sequenceing for range and upload controls (FF) 
4.6.3.c event sequencing for select/select1 controls (focus changes) (EMC)
4.6.5.a event sequencing for submit controls (EMC)
4.8.1.a getInstanceDocument() method (EMC)
4.8.1.b getInstanceDocument() throws a DOMException (EMC) 
Recommended:
4.2.2.c2 invalid QNAME causes xforms-binding-exception (EMC)
4.2.4.a xforms-model-destruct dispatched after form submission (EMC)

>From chapter 5:
Required:
5.1.a built-in primitive types (FF) 
5.1.c supported primitive XML schema types in basic processors (FF)
5.2.1.a XForms datatypes to allow empty content - built-in primitive types 
(FF)
5.2.1.b XForms datatypes to allow empty content - built-in derived types 
(FF) 
5.2.1.c built-in data types which allow empty content for basic processors 
(FF)
5.2.2.a listItem datatype (FF)
5.2.3.a listItems datatype (FF)
5.2.4.a dayTimeDuration datatype (FF) 
5.2.5.a yearMonthDuration datatype (FF)
5.2.6.a email datatype (FF)
5.2.7.a card-number datatype (FF)

>From chapter 6:
Required:
6.2.1.a atomic datatype 

>From chapter 7:
Required:
7.8.3.a digest() function using sha1, md5, and sha256 (EMC)
7.8.3.b digest() function using sha384 and sha512 (EMC)
7.8.4.a hmac() function using sha1, md5, and sha256 (EMC)
7.8.4.b hmac() function using sha384 and sha512 (EMC)
7.10.3.a id() function (FF)
7.11.2.b event() function with resource-uri property (FF)

>From chapter 8:
Required:
8.1.a navindex and accesskey (FF)
8.1.1.a form control violates data binding restriction (FF)
8.1.1.b non-relevant form control becoming relevant (EMC)
8.1.6.a mediatype attribute for upload element (EMC)
8.1.6.d data binding restrictions for upload element (EMC)
8.1.7.d incremental attribute of range element (FF)
8.1.7.f data binding restrictions on range element (FF)
8.1.7.g data binding restrictions on range element for basic processors 
(FF)
8.1.10.a selection attribute of select element (FF)
8.1.10.c appearance attribute of select element (EMC)
8.1.10.d value out of range of a select element (EMC)
8.2.3.b hint element uses inline text (EMC)
8.3.2.a item element (EMC)
8.3.3.b precedence for value element (EMC)

>From chapter 9:
Recommended:
9.1.1.a2 group element inside case element of a switch (FF)
9.2.1.a2 switch element receives events (FF)
9.3.5.a repeating structures via attributes Fail (EMC) 
9.3.6.a itemset element example (EMC)
9.3.7.b copy element has xforms-binding-exception event 

>From chapter 10:
Required:
10.3.d insert action using at attribute (FF)
10.3.e insert action using position attribute (EMC)
10.3.h insert action and repeat element (FF)
10.3.j insert action - copying an attribute (FF)
10.7.1.a setfocus element with value child element Failed (FF)
10.7.1.b control element precedence tests (FF)
10.8.d dispatch element dispatches custom event that bubbles (EMC)
10.8.f dispatch element dispatches cancelled predefined event (EMC)
10.8.1.c value attribute has precedence in name element (FF)
10.8.2.c value attribute has precedence in target element (FF) 
10.8.3.c value attribute has precedence in delay element (FF)
10.18.a iteration of XForms actions (EMC)
10.18.b iteration of XForms actions using action element (EMC)
10.18.d XForms actions with if and while attributes (EMC)
10.18.e iteration of XForms actions - Summing Selected Results example 
(EMC)
Recommended:
10.6.1.a case element child of the toggle element (FF)
10.14.1.a load element with resource child element Failed (FF)
10.14.1.b value attribute has precedence in load element (FF)
10.15.a send element (EMC)

>From chapter 11:
* Submission Module is Recommended
11.1.i relevant attribute of submission element (EMC)
11.1.m mediatype attribute of submission element (EMC)
11.1.n encoding attribute of submission element (EMC)
11.1.q cdata-section-elements attribute of submission element (EMC)
11.1.r replace attribute of submission element (EMC)
11.1.s1 instance attribute of submission element (EMC)
11.1.s2 invalid instance attribute of submission element (EMC)
11.1.t target attribute of submission element (EMC)
11.1.u separator attribute of submission element (EMC)
11.1.v includenamespaceprefixes attribute of submission element (EMC)
11.2.b non-relevant nodes are not serialized (EMC)
11.2.c xforms-submit with empty instance data (EMC)
11.2.e xforms-submit stops with invalid instance data (EMC)
11.3.a xforms-submit-serialize event (EMC)
11.3.b xforms-submit-serialize event with submission-body property (EMC)
11.4.b accessing context information for xforms-submit-done event (EMC)
11.5.a xforms-submit-error event (EMC)
11.5.b accessing context information for xforms-submit-error event (EMC)
11.6.1.a resource element (EMC)
11.6.1.b resource element with value attribute (EMC)
11.8.a header element of submission element (EMC)
11.8.b header element with nodeset attribute (EMC)
11.8.1.a name element with value attribute (EMC)
11.8.2.a value element with value attribute (EMC)
11.9.d HTTP multipart-post as a submission option (EMC)
11.9.e HTTP form-data-post as a submission option (EMC)
11.9.f HTTP urlencoded-posts as a submission option (EMC)
11.9.j HTTPS multipart-post as a submission option (EMC)
11.9.k HTTPS form-data-post as a submission option (EMC)
11.9.l HTTPS urlencoded-posts as a submission option (EMC)
11.9.o File Put as a submission option (EMC)
11.9.1.a get submission method (EMC)
11.9.2.b multipart-post submission method (EMC)
11.9.3.b put submission method to local file (EMC)
11.9.5.a serialization as application/xml (EMC)
11.9.6.a serialization as multipart/related (EMC)
11.9.7.a serialization as multipart/form-data (EMC)
11.9.8.a serialization as application/x-www-form-urlencoded (EMC)
11.10.c submission response with target receiving instance (EMC)
11.11.2.a indicating a SOAP submission (EMC)
11.11.3.b SOAP HTTP binding with post method (EMC)
11.11.3.c SOAP HTTP binding (EMC)

[1] http://code.google.com/p/ubiquity-xforms/
[2] http://www.w3.org/MarkUp/Forms/specs/XForms1.1/index-diff.html#conform

Keith