Re: Update on MSE and EME Testing

Jerry,

There are a few things which need to be fixed up before we can really read
anything into the test results:

   - There is a bug in util/utils.js in getSupportedKeysystem() which
   returns com.widevine.alpha for Edge, meaning that playback tests on Edge
   all fail. I presume you fixed this before generating the results.
   - The "less than 2" script should not consider two versions of Chrome or
   two versions of Firefox as 2 versions passing
   - The test names for the drm tests include the keysystem. So, when you
   have it working with appropriate keysystems on each browser, they will not
   match up correctly on the test-results page. We'll need to remove the
   keysystem name from the test name and just put "drm"
   - The test names for the polyfill tests should probably start "Polyfill"
   for readability

...Mark

On Fri, Aug 5, 2016 at 1:44 PM, Jerry Smith (WPT) <jdsmith@microsoft.com>
wrote:

> I’ve made an initial posting of test results on a few browsers (Edge, two
> versions of Chrome and two also of Firefox).  It’s a bit messy, because it
> includes Google subfolder tests for Clear-Key (dups) and also tests with
> and without some polyfills Mark has been developing.
>
>
>
> You can see current EME results using these links:
>
>
>
> •                     http://w3c.github.io/test-
> results/encrypted-media/all.html
>
> •                     http://w3c.github.io/test-results/encrypted-media/
> complete-fails.html
>
> •                     http://w3c.github.io/test-
> results/encrypted-media/less-than-2.html
>
>
>
> I plan to pull down the older versions of Chrome and Firefox soon, but
> wanted to be sure the deltas between the two make sense.  Comments are
> welcome.
>
>
>
> For convenience, here are the MSE links:
>
>
>
> •                     http://w3c.github.io/test-
> results/media-source/all.html
>
> •                     http://w3c.github.io/test-
> results/media-source/complete-fails.html
>
> •                     http://w3c.github.io/test-results/media-source/less-
> than-2.html
>
>
>
> Jerry
>
>
>
> *From:* Mark Watson [mailto:watsonm@netflix.com]
> *Sent:* Friday, August 5, 2016 12:44 PM
> *To:* Henri Sivonen <hsivonen@hsivonen.fi>
> *Cc:* Chris Pearce <cpearce@mozilla.com>; Jerry Smith (WPT) <
> jdsmith@microsoft.com>; public-html-media@w3.org; jyavenard@mozilla.com;
> Matthew Wolenetz <wolenetz@google.com> (wolenetz@google.com) <
> wolenetz@google.com>; Anthony Jones <ajones@mozilla.com>; Philippe Le
> Hegaret (plh@w3.org) <plh@w3.org>; Paul Cotton <Paul.Cotton@microsoft.com>
> *Subject:* Re: Update on MSE and EME Testing
>
>
>
> We have been working on additional tests for the Encrypted Media
> Extensions specification.
>
>
>
> Unfortunately, it has been very difficult to get the tests we have been
> working to work, due to lack of the necessary server support and
> uncertainty as to what is and is not implemented in  browsers. At this
> stage, therefore, we do not yet have DRM tests for persistent-license or
> persistent-usage-record session types.
>
>
>
> There are two outstanding pull requests:
>
> - https://github.com/w3c/web-platform-tests/pull/3424 adds support for
> persistent-license (clearkey)
>
> - https://github.com/w3c/web-platform-tests/pull/3417 adds a
> waitingforkey test
>
>
>
> I expect we will make further progress over the coming week (I will be
> out, but Sukmal will have some time to work on this).
>
>
>
> ...Mark
>
>
>
>
>
> On Fri, Aug 5, 2016 at 7:50 AM, Mark Watson <watsonm@netflix.com> wrote:
>
>
>
>
>
> On Fri, Aug 5, 2016 at 1:20 AM, Henri Sivonen <hsivonen@hsivonen.fi>
> wrote:
>
> On Thu, Aug 4, 2016 at 7:26 AM, Mark Watson <watsonm@netflix.com> wrote:
> > I believe that persistent-usage-record functionality works with Firefox /
> > Primetime, with some rough edges on the Primetime side. However, we do
> not
> > have a Primetime server for our testing, so we cannot verify this.
>
> While some initial steps (that I've referred to previously on this
> list) were taken towards persistent-usage-record -like functionality
> on the Firefox side, those steps never reached the point of supporting
> the persistent-usage-record API. As Chris said, we have no plans to
> implement "persistent-usage-record" sessions.
>
> > The Edge and CastTV implementations are close to compliance, I believe,
> and we are working on tests that will demonstrate that.
>
> Do those implementations write to plain disk-like storage on CDM
> shutdown or do they use some form of tamper-resistant storage during
> playback? Which Key System does CastTV use? Was the text in the spec
> alone sufficiently detailed to reach interop?
>
>
>
> ​I don't know the implementation details. They are both PlayReady. They
> don't reach interop. When I say "close to compliance", I mean that with a
> bit of JS help you can patch up the non-compliances. At least, that is my
> hope at present: we don't have everything working yet.
>
>
>
> ...Mark
>
>
>
>
> --
> Henri Sivonen
> hsivonen@hsivonen.fi
> https://hsivonen.fi/
>
>
>
>
>