- From: Francois Daoust <fd@w3.org>
- Date: Wed, 31 Aug 2016 12:06:29 +0200
- To: "'Robin Berjon'" <robin@berjon.com>, "'Michael [tm] Smith'" <mike@w3.org>, "'Jerry Smith \(WPT\)'" <jdsmith@microsoft.com>
- Cc: "'Philippe Le Hegaret'" <plh@w3.org>, <public-html-media@w3.org>, "'Paul Cotton'" <Paul.Cotton@microsoft.com>
> De : Robin Berjon [mailto:robin@berjon.com] > > On 22/08/2016 04:13 , Francois Daoust wrote: > > > Changing the incriminated line to "if (haveResults < 1) continue;" would > > fix the problem. However, that line comes with a comment "skip tests > > with all but one undefined" and this check was first introduced in May > > 2014 in a commit entitled "filter out broken results": > > > https://github.com/w3c/wptreport/commit/6c9c20545aa3230bff6f165c1e129 > f8b89dd53df > > > > In other words, this behavior seems intended, although I don't > > understand the rationale. Robin, do you remember why you had to do > that? > > That's from a very long time ago. I know I had to do a fair amount of > data munging because of false negatives due to timeouts that really > shouldn't have timed out back then, due to issues with the test runner > and the performance of browsers under it. > > I wouldn't be surprised if this were no longer needed. Hey, just remove > that line and see if anything breaks :) Good :) So I dug into the code of wptreport a bit more and prepared a pull request that completely drops the "data munging" mechanism: https://github.com/w3c/wptreport/pull/10 I checked with test results for different specs, that does not seem to break anything, and that fixes the problem for the encrypted-media test suite (in particular, subtests that have timeout results across agents correctly appear in "complete-fails.html"). Thanks, Francois.
Received on Wednesday, 31 August 2016 10:06:45 UTC