W3C home > Mailing lists > Public > public-html-media@w3.org > October 2016

RE: Updated EME Test Results

From: Jerry Smith (WPT) <jdsmith@microsoft.com>
Date: Tue, 11 Oct 2016 00:43:04 +0000
To: Mark Watson <watsonm@netflix.com>
CC: David Dorwin <ddorwin@google.com>, Paul Cotton <Paul.Cotton@microsoft.com>, Matt Wolenetz <wolenetz@google.com>, Philippe Le Hégaret <plh@w3.org>, "public-html-media@w3.org" <public-html-media@w3.org>
Message-ID: <BY2PR03MB041D22E7247DB0A64E4C1A2A4DA0@BY2PR03MB041.namprd03.prod.outlook.com>
Some breakdown:


-          Wrong error type:                        24 failures

-          Wrong promise handling:            14 failures

That’s close to half of the total failures.

There are about 19 test timeouts, which is also a large bucket.

As previously, there are 14 “failures” that aren’t real.  They have one passing implementation because they are PlayReady keySystem specific.

Jerry

From: Jerry Smith (WPT)
Sent: Monday, October 10, 2016 1:44 PM
To: 'Mark Watson' <watsonm@netflix.com>
Cc: David Dorwin <ddorwin@google.com>; Paul Cotton <Paul.Cotton@microsoft.com>; Matt Wolenetz <wolenetz@google.com>; Philippe Le Hégaret <plh@w3.org>; public-html-media@w3.org
Subject: RE: Updated EME Test Results

I’ve posted updated test results, but haven’t had a chance to go through them in detail.  I intend to do that by end of today.

This is the first test pass that has all Google tests migrated fully, and should represent the majority of our planned test cases.


-          http://w3c.github.io/test-results/encrypted-media/all.html


o   Test files: 97; Total subtests: 320



-          http://w3c.github.io/test-results/encrypted-media/less-than-2.html


o   Test files without 2 passes: 42; Subtests without 2 passes: 83; Failure level: 83/320 (25.94%)



-          http://w3c.github.io/test-results/encrypted-media/complete-fails.html


o   Completely failed files: undefined; Completely failed subtests: 45; Failure level: 45/320 (14.06%)

Jerry
Received on Tuesday, 11 October 2016 00:43:36 UTC

This archive was generated by hypermail 2.3.1 : Tuesday, 11 October 2016 00:43:37 UTC