- From: Karl Tomlinson <karlt+public-audio@karlt.net>
- Date: Tue, 09 Jun 2015 13:42:30 +1200
- To: Joe Berkovitz <joe@noteflight.com>
- Cc: Raymond Toy <rtoy@google.com>, Chris Lowis <chris.lowis@gmail.com>, Paul Adenot <padenot@mozilla.com>, Audio Working Group <public-audio@w3.org>
Joe Berkovitz writes: >> My assumption here was that, say, FF found a bug in their implementation >> and want a test for that and pushes that to the repo. Which then causes >> Chrome to fail that test (for whatever reason). We wouldn't want to block >> FF from having a regression test, but we wouldn't want to be blocked from >> updating because the test will cause failures in our test bots. >> > > Perhaps we need a way to gate tests in the suite so that you are able to > arbitrarily exclude ones that are not applicable for some reason (or > arbitrarily include the ones you want). Web platform tests are run on Gecko using wptrunner. This provides a mechanism [1] to annotate the current result of the test or to disable if the result is not deterministic. [1] http://wptrunner.readthedocs.org/en/latest/expectation.html Sometimes tests pulled into Gecko after development against another browser identify bugs in Gecko, which can later be fixed, and Gecko sometimes identifies bugs in the tests, but it is not difficult to deal with these, and it is beneficial to see where such differences exist.
Received on Tuesday, 9 June 2015 01:44:01 UTC