Re: wptreport issue

Jerry, Mike, Robin,

On 20/08/2016 07:39, Michael[tm] Smith wrote:
> +Robin Berjon
>
> "Jerry Smith (WPT)" <jdsmith@microsoft.com>, 2016-08-19 22:30 +0000:
>> Archived-At: <http://www.w3.org/mid/BY2PR03MB04156852535B860F021498EA4160@BY2PR03MB041.namprd03.prod.outlook.com>
>>
>> I'm trying to track down an issue I'm hitting using wptreport.  EME has
>> JSONs that show results for specific tests where the report pages just
>> list the test itself with no results.  Any ideas on how to fix this?
>>
>> An example is the /encrypted-media/Google/encrypted-media-keystatus.html
>> test:
>> https://rawgit.com/w3c/test-results/gh-pages/encrypted-media/all.html#test-file-9.
>> Current JSONs include some timeout and passing results for it.
>
> I looked at the JSON files today and tried to troubleshoot it but still
> have no clue either.

In wptreport, the line that skips the subtests is in index.js, L187:
https://github.com/w3c/wptreport/blob/master/index.js#L187

Subtest results only appear provided the JSON files contain at least two results that are different from TIMEOUT and NOTRUN. In the provided example, only CH54.json contains a PASS result for this substest, the other JSON files have TIMEOUT results.

Changing the incriminated line to "if (haveResults < 1) continue;" would fix the problem. However, that line comes with a comment "skip tests with all but one undefined" and this check was first introduced in May 2014 in a commit entitled "filter out broken results":
https://github.com/w3c/wptreport/commit/6c9c20545aa3230bff6f165c1e129f8b89dd53df

In other words, this behavior seems intended, although I don't understand the rationale. Robin, do you remember why you had to do that?

Thanks,
Francois.

Received on Monday, 22 August 2016 08:13:17 UTC