[selectors-api] Updated Testsuite

Hi,
   I worked on redesigning the testsuite to resolve a number of issues 
with the design of the old tests, including removing significant amount 
of redundancy and fixing broken tests.  In the process, I have 
discovered a number of new bugs in various browsers that need to be 
addressed.

*Summary of Changes*

1. The Level 1 baseline tests are no longer a subset of the additional 
tests. The baseline and additional test files now test a completely 
separate set of tests.

2. The :target tests were moved from baseline to additional because 
:target was added in Selectors 3.

3. All selectors from CSS 1, 2 and Selectors 3 are being tested.

4. level1-lib.js now contains the common functions shared among the 3 
testsuites: baseline, additional and xhtml.

5. The files level1-content.html and level1-content.xht contain the all 
of the (X)HTML that is queried by the API, included within the iframe of 
each testsuite.  Every element in the file is assigned an ID that is 
used for verification.

6. selectors.js contains all selectors for both valid and invalid 
selector tests.  This includes annotations about the expected result 
from each selector, with a list of expected element IDs.  These IDs are 
compared with the results from the queries.

This change in architecture resolves some major issues with the old 
testsuite, which had previously resolved in false positives being 
reported.  Notably, the namespace tests. See below.

Some additional tests were added for :link and :visited, which now 
reveal failures in both Gecko and WebKit.

Notes:
* The Selectors API Level 2 tests have not yet been updated.
* Some tests are still needed for :not() and ~ sibling combinator.
* Some test descriptions have not been finished in selectors.js.


*Failures*

Opera:

Baseline: 1157 PASS, 10 FAIL
* "" (no parameter) tests (throwing WRONG_ARGUMENTS_ERR instead of 
TypeError)
* "#head :link, #head :visited" (new bug in 12.10 beta, passes in 12.02)

Additional: 512 PASS, 0 FAIL


Chrome/Safari:

Baseline: 1140 PASS, 27 FAIL
* hasFeature() returns false
* namespace tests

Additional: 508 PASS, 4 FAIL
* :target tests


Mozilla Gecko:

Baseline: 1154 PASS, 13 FAIL
* hasFeature() returns false
* "" (no parameter) tests (throwing NS_ERROR_XPC_NOT_ENOUGH_ARGS instead 
of TypeError)
* :link and :visited for fragment and detached element tests failing to 
match A and AREA elements.

Additional: 512 PASS, 0 FAIL
* No failures

(I haven't got access to IE at this time, so I have not tested it.)

The testsuite can be found here.
http://dev.w3.org/2006/webapi/selectors-api-testsuite/

All files for these tests are located in the hg repo.
http://w3c-test.org/webapps/SelectorsAPI/tests/submissions/Opera/

-- 
Lachlan Hunt - Opera Software
http://lachy.id.au/
http://www.opera.com/

Received on Monday, 22 October 2012 10:11:12 UTC