RE: Updated EME Test Status

We have a dialog going on the DRMToday account.  I’ve asked them to set up accounts in Mark’s name and mine.

Jerry

From: David Dorwin [mailto:ddorwin@google.com]
Sent: Tuesday, September 13, 2016 1:57 PM
To: Jerry Smith (WPT) <jdsmith@microsoft.com>
Cc: Mark Watson <watsonm@netflix.com>; Paul Cotton <Paul.Cotton@microsoft.com>; Matt Wolenetz <wolenetz@google.com>; <public-html-media@w3.org> <public-html-media@w3.org>
Subject: Re: Updated EME Test Status

Migrating to the new account may help: https://github.com/w3c/web-platform-tests/issues/3624. Who has the account information and can do that?

On Tue, Sep 13, 2016 at 1:38 PM, Jerry Smith (WPT) <jdsmith@microsoft.com<mailto:jdsmith@microsoft.com>> wrote:
Mark:  Please confirm whether you have a current contact to troubleshoot DRMToday.com license server issues.  I believe you don’t.

From: David Dorwin [mailto:ddorwin@google.com<mailto:ddorwin@google.com>]
Sent: Tuesday, September 13, 2016 11:40 AM
To: Mark Watson <watsonm@netflix.com<mailto:watsonm@netflix.com>>
Cc: Jerry Smith (WPT) <jdsmith@microsoft.com<mailto:jdsmith@microsoft.com>>; Paul Cotton <Paul.Cotton@microsoft.com<mailto:Paul.Cotton@microsoft.com>>; Matt Wolenetz <wolenetz@google.com<mailto:wolenetz@google.com>>; <public-html-media@w3.org<mailto:public-html-media@w3.org>> <public-html-media@w3.org<mailto:public-html-media@w3.org>>
Subject: Re: Updated EME Test Status

Thanks, Jerry. Yes, it looks like a license server issue:
https://lic.staging.drmtoday.com/license-proxy-widevine/cenc/ Failed to load resource: the server responded with a status of 403 (Forbidden)

On Tue, Sep 13, 2016 at 11:36 AM, Mark Watson <watsonm@netflix.com<mailto:watsonm@netflix.com>> wrote:
Jerry, all,

There's definitely something wrong here. I expect at least least "drm, temporary, mp4, playback, setMediaKeys after updating session" (and many others) to pass on all three browsers and it is showing as a complete fail. I'll check whether it is working for me here. Perhaps there has been some regression.

...Mark

On Tue, Sep 13, 2016 at 11:24 AM, Jerry Smith (WPT) <jdsmith@microsoft.com<mailto:jdsmith@microsoft.com>> wrote:
I’ve posted updated results for EME tests:


-          http://w3c.github.io/test-results/encrypted-media/all.html


o   Test files: 70; Total subtests: 220

-          http://w3c.github.io/test-results/encrypted-media/less-than-2.html


o   Test files without 2 passes: 44; Subtests without 2 passes: 64; Failure level: 64/220 (29.09%)

-          http://w3c.github.io/test-results/encrypted-media/complete-fails.html


o   Completely failed files: 44; Completely failed subtests: 29; Failure level: 29/220 (13.18%)

Some comments about these results:


1.       They don’t filter for single valid test outcomes (either pass or fail), and timeouts as failures.  Most of the complete-fails are test timeouts.

2.       Test cases have only been partially (~25%) migrated to full drm from Clear Key versions contributed by Google.  The original Google tests are run if not migrated, and can be distinguished by “Google” in the test file path.

Jerry