web-platform-tests now running in Gecko CI system

As of Friday the full [1] set of web-platform-tests are now running in
the Mozilla Continuous Integration (CI) system [2]. This means that they
are running on every single commit to the main Gecko development tree,
and that developers are responsible for keeping the tests from regressing.

At present they are only enabled on Linux 64, but they will be enabled
on other platforms once its clear that doing so isn't going to introduce
stability problems [3]. Based on a few days worth of data, the test
stability seems to be tolerable, but not perfect, and we have already
had some success in fixing bugs caught by the tests.

Obviously there is still a lot of work to be done — not least getting
more Mozilla-authored tests submitted to web-platform-tests — but I'd
nevertheless like to thank all the people who have helped us get this
far through contributions to wptrunner, wptserve, the documentation at
testthewebforward.org, and, of course, to the tests themselves. This has
very much been a community effort, and I hope that by that as a
community we can make a lasting positive impact on the web platform.

[1] Excluding reftests that are simply not enabled yet, and a relatively
short list of tests that are disabled for stability reasons; I will post
these in a separate mail.
[2] https://tbpl.mozilla.org/?tree=Mozilla-Inbound&jobname=web-platform-*
[3] Linux, Windows XP, Windows 7 and Windows 8 could probably be enabled
immediately, OSX seems to be more troublesome, and the harness currently
doesn't support running on Android.

Received on Monday, 8 September 2014 15:19:25 UTC