Re: Unstable tests

On 25/03/14 11:15, James Graham wrote:
> Using data from a set of web-platform-tests runs in desktop Firefox, I
> have a list of tests that aren't producing consistent results. This data
> is based on 10 runs per platform for 5 different platforms (2 Linux, 3
> OS X). Instability is considered per-platform (so a test that
> consistently produces one result on OSX and another on Linux is
> considered stable). The list of tests that produced unstable results in
> any configuration is at [1]. For the purposes of presentation I squashed
> the results down into a single list without platform information, but I
> can change that if needed.

With some work on test stability, I now have a new list [1]. This is 
from 45 runs across 4 different platforms, so it won't find tests that 
are only rarely unstable (roughly speaking this should be about 90% of 
tests that change results at least one time in 20).

Help fixing the remaining tests, or identifying gecko bugs, is welcome, 
as always.

[1] http://hoppipolla.co.uk/410/unstable-1.txt