Re: CfC - publish Selectors API as CR

Maciej Stachowiak wrote:
>> The proposed exit criteria are in a separate thread, but essentially are:
>>
>> For a set of tests based on HTML, CSS 2.1 selectors and this spec,
>> there are two implementations that pass every test interoperably, and
>> do not fail any "additional" tests based on misimplementing this
>> specification (i.e. failures based on not supporting a technology used
>> only in the additional tests, such as MathML, will not be taken into
>> account).
>
> Request for clarification. Does this require:
>
> A) There must be two implementations, each of which passes every test
> (i.e. the same two implementations pass all the tests); or
> B) For each test, there are two implementations that pass it (but not
> necessarily the same two for every test).
>
> It reads like (A), but I have seen similar wording interpreted as (B) in
> the context of other specs...

The intention in the original exit criteria proposal [1] was for there 
to be at least two complete implementations, each passing 100% of the 
baseline tests.  I can make this clearer in the exit criteria as follows:

---

There must be at least two complete, independent implementations, each 
of which must pass 100% of the baseline testsuite and should pass 
additional tests, dependent on the following conditions:

* The implementations must be native implementations in shipping
   products.  (JavaScript library implementations don't count).

* The baseline testsuite comprises tests that check for conformance to
   all requirements in the API using only HTML and Selectors defined in
   CSS 2.1.

* Tests using Selectors introduced in Selectors Level 3, or XHTML+SVG,
   are considered to be additional tests.

* An additional test may be marked as N/A for an implementation if:
   - The test uses a selector that the implementation does not support
   - The test uses XHTML+SVG that the implementation does not support

* Implementations are not required to pass all additional tests,
   however no failures must be caused by an incorrect implementation of
   the API itself. Failures of additional tests caused only by an
   incorrect implementation of Selectors do not count.

---

The current state of implementations is as follows:

Minefield:
   Baseline Tests:   HTML/CSS2.1:    PASS
   Additional Tests: HTML/CSS3:      PASS
   Additional Tests: XHTML+SVG/CSS3: PASS

Opera gogi (Internal build)
   Baseline Tests:   HTML/CSS2.1:    PASS
   Additional Tests: HTML/CSS3:      FAIL 4 (non-API bugs)
   Additional Tests: XHTML+SVG/CSS3: FAIL 22 (non-API bugs)

WebKit:
   Baseline Tests:   HTML/CSS2.1:    FAIL 16 (API bug)
   Additional Tests: HTML/CSS3:      FAIL 16
   Additional Tests: XHTML+SVG/CSS3: FAIL 16

IE8:
   Baseline Tests:   HTML/CSS2.1:    FAIL 252 (API bugs)
   Additional Tests: HTML/CSS3:      N/A
   Additional Tests: XHTML+SVG/CSS3: N/A

BlackBerry 9700 browser:
   (Kartikaya Gupta from RIM e-mailed me off list about this to tell me,
    I'm unable to verify these results myself without access to the
    device.)
   Baseline Tests:   HTML/CSS2.1:    PASS
   Additional Tests: HTML/CSS3:      PASS
   Additional Tests: XHTML+SVG/CSS3: PASS

With Minefield and BlackBerry, we have two complete implementations 
passing everything.  Opera's results also meet the above criteria, so 
that gives us 3 implementations.

[1] http://lists.w3.org/Archives/Public/public-webapps/2009AprJun/1221.html

-- 
Lachlan Hunt - Opera Software
http://lachy.id.au/
http://www.opera.com/

Received on Thursday, 26 November 2009 14:06:03 UTC