RE: Call for Consensus - Selectors API to Candidate Rec

I went through the test results in IE8 just to see what the breakdown was and thought I'd pass this info along. I appreciate the thoroughness of these tests, though it bothers me a bit that we get hammered because of the various WebIDL binding issues (e.g., IE8's exception can't be mapped to DOMException.SYNTAX_ERR). Having said that, I understand the desire to be precise and test not only the Selectors API but also the WebIDL binding dependencies that go with it.

Current pass rate: 47.4%

Best possible current pass rate (excluding WebIDL issues): 1241 tests (57.3%)

Best possible pass rate (excluding bugs/WebIDL issues): 1539 tests (71.1%)

I'd like to say that 29.9% of the remaining tests are above and beyond the CSS2.1 spec's defined selectors--however, IE8 supports _some_ of the CSS3 selectors already, so it's not such a clean division of the percentages :-)

John, thanks again for authoring these tests. To all: I'm still working on collecting IE8 internal test cases and evaluating their suitability for the official test suite. Thanks for your patience--I hope to get back to you in a week or two (been pretty busy here).

-Travis

Analysis below
----

1 - Actual Selectors API bugs
====================================================================
288 tests (13.3%)

(48 tests) Passing null/undefined/[nothing] to querySelector/querySelectorAll (we don't throw an exception here but should). Note, throwing an exception would move these tests into the "WebIDL binding dependencies" category based on the test case design.

(240 tests) Whitespace trimming: removing superfluous whitespace before/after the selector. The tests include spaces [ ], tabs [\t], newlines [\n], and carriage returns [\r].

2 - Tests IE8 cannot pass due to dependency on WebIDL bindings
====================================================================
216 tests (9.9%)

(8 tests) An "Interface" check fails (says we don't support querySelector/All) because "typeof obj.querySelector" is "object" not "function" -- this is related to general function bindings for Jscript which are not well supported now.

(16 tests) Empty string tests fail (we correctly throw an exception, we just don't support the DOMException WebIDL binding with constant value SYNTAX_ERR). NOTE: We do support the "Error" interface object, but not the constants...

(192 tests) Negative tests (that verify that we fail when we should fail)-- we throw exceptions as expected, but fail the test per the reason listed previously (not supporting the DOMException WebIDL binding with constant value SYNTAX_ERR).

3 - Tests IE8 rightfully fails
====================================================================
657 tests (30.3%)

(10 tests) Failures due to Bad existing CSS selector support
  - Element.querySelectorAll: .attrStart > .t3[align^=""] (5)
  - Element.querySelectorAll: .attrEnd > .t3[align$=""] (5)

(647 tests) Failures due to unsupported CSS3 Selectors
  - :root (18)
  - :nth-last-child (58)
  - :nth-child (108)
  - :nth-of-type (42)
  - :nth-last-of-type (56)
  - :not (66 unique, 156 in combination w/other non-supported selectors)
  - :only-of-type (66)
  - :last-child (35)
  - :first-of-type (28)
  - :last-of-type (28)
  - :only-child (30)
  - :empty (41)
  - :enabled (14)
  - :disabled (15)
  - :checked (29)
  - :target (13)



-----Original Message-----
From: public-webapps-request@w3.org [mailto:public-webapps-request@w3.org] On Behalf Of John Resig
Sent: Friday, February 13, 2009 6:33 AM
To: Lachlan Hunt
Cc: Charles McCathieNevile; WebApps WG
Subject: Re: Call for Consensus - Selectors API to Candidate Rec


> Firefox appears to have some issues that might related to the API, though I
> haven't investigated the cause of those yet, so I don't know for sure.
>  Unfortunately, IE8 can't run John's tests because it relies on some DOM2
> features that aren't yet supported, so the testing framework would need to
> be rewritten to make it work.

Just to update: The test suite now runs in IE 8. Demo:
http://ejohn.org/apps/selectortest/

As of last night's nightly WebKit has the first 100% passing implementation.

--John

Received on Thursday, 26 February 2009 20:01:25 UTC