wptrunner and IE

Hello public-test-infra,

As a bootstrap effort to get W3C tests up and running for IE, a team at Microsoft recently prototyped a rudimentary w-p-t automation and reporting system (for testharness cases, using Selenium WebDriver on BrowserStack). This is what we used to contribute [1][2] IE11 results for the recent html [3] and dom [4] implementation reports.

In an effort to get a preliminary understanding of browser interoperability across the w-p-t suite, we managed to hook up our runner to Firefox and Chrome as well (on Windows 8.1).  FWIW, our full runs have been stabilizing around 191k tests, and our cross-browser results for the html (142k tests) and dom (47k tests) suites generally vary from the recent W3C test runner results [5][6] by only several tenths of a percent.

As a bootstrap effort to get W3C tests up and running, our system was put together quickly (it has some hacks and tight coupling) and uses a lot of msft tech (IIS, ASP.NET). Now we're looking to refactor our system to use and contribute to the central W3C test infra wherever possible. Here is a PR  [7] for wptserve that we found Windows to need for opening binary files, and a few test bug fixes [8].

James-I'd like to introduce you to Anton, who will be looking into making wptrunner work for IE with our new WebDriver implementation. We will send you (and public-test-infra) our questions as we start working on it.

Thanks,
Erika

[1] https://github.com/w3c/html/commit/dd718c650eb596d5a29e9949ccff057fb39f1820   
[2] https://github.com/w3c/dom/commit/84a4bc9133585a47031b3a4c69fbc43da9a57105   
[3] http://w3c.github.io/html/test-results/less-than-2.html   
[4] http://w3c.github.io/dom/test-results/less-than-2.html   
[5] http://w3c.github.io/html/test-results/all.html   
[6] http://w3c.github.io/dom/test-results/all.html   
[7] https://critic.hoppipolla.co.uk/r/870  
[8] https://critic.hoppipolla.co.uk/r/1411

Received on Tuesday, 29 April 2014 06:21:26 UTC