Re: FW: Update on MSE and EME Testing

Hi Jerry,

Regarding EME, we have no plans to implement "persistent-usage-record"
sessions. You mention two implementations which are "close to passing",
which implementations are those?

We have a patch for waitingforkey that apparently works, we've just not
gotten around to cleaning it up and landing it. We hadn't prioritized it
until Joey Parish from the Shaka Player complained about it
<https://bugzilla.mozilla.org/show_bug.cgi?id=1145011#c1>. And then Shaka
Player went and removed their use of "waitingforkey" altogether and complained
they couldn't tell when they weren't waiting for a key anymore
<https://github.com/w3c/encrypted-media/issues/284>... So I'd like to hear
Joey answer my comment in that issue before landing our patch.

So we could land our waitingforkey patch tomorrow if we wanted, but it's
not clear to me that the authors who wanted that feature want it in its
current form. I wouldn't object if we dropped waitingforkey.

FYI, I am working towards getting Google's Web Platform EME Tests passing
in Firefox 51 Nightly. I've fixed a few issues so far. Most of the problems
I've found so far are bugs like us not throwing the specified exception
code, or parser bugs. I hope to have them passing or debugged in the next
few weeks. I will file issues against the tests if I find problems. I've
filed issue 3430 <https://github.com/w3c/web-platform-tests/issues/3430> so
far.


Cheers,
Chris Pearce.


On Thu, Aug 4, 2016 at 2:43 AM, Paul Cotton <Paul.Cotton@microsoft.com>
wrote:

> Forwarding this thread to public-html-media@w3.org with permission.
>
>
>
> This thread is about Firefox’s implementation of specific MSE features and
> their comments on the MSE test suite.
>
>
>
> /paulc
>
>
>
> *From:* jyavenard@mozilla.com [mailto:jyavenard@mozilla.com]
> *Sent:* Tuesday, August 2, 2016 10:11 PM
>
> *To:* Paul Cotton <Paul.Cotton@microsoft.com>; Jean-Yves Avenard <
> jya@mozilla.com>; Jerry Smith (WPT) <jdsmith@microsoft.com>
> *Cc:* Chris Pearce <cpearce@mozilla.com>; Matt Wolenetz <
> wolenetz@google.com>; Anthony Jones <ajones@mozilla.com>; Philippe Le
> Hegaret (plh@w3.org) <plh@w3.org>
> *Subject:* RE: Update on MSE and EME Testing
>
>
>
> Hello
>
>
>
> Here they are:
>
> https://github.com/w3c/web-platform-tests/issues/created_by/jyavenard
>
>
>
> Sent from Mail <https://go.microsoft.com/fwlink/?LinkId=550986> for
> Windows 10
>
>
>
> *From: *Paul Cotton <Paul.Cotton@microsoft.com>
> *Sent: *Wednesday, 3 August 2016 10:07 PM
> *To: *jyavenard@mozilla.com; Jean-Yves Avenard <jya@mozilla.com>; Jerry
> Smith (WPT) <jdsmith@microsoft.com>
> *Cc: *Chris Pearce <cpearce@mozilla.com>; Matt Wolenetz
> <wolenetz@google.com>; Anthony Jones <ajones@mozilla.com>; Philippe Le
> Hegaret (plh@w3.org) <plh@w3.org>
> *Subject: *RE: Update on MSE and EME Testing
>
>
>
> > I believe I have opened some bugs on web-platform-tests GitHub ; but no
> progresses were made.
>
>
>
> Could you please provide pointers to those bugs?
>
>
>
> /paulc
>
>
>
> *From:* jyavenard@mozilla.com [mailto:jyavenard@mozilla.com
> <jyavenard@mozilla.com>]
> *Sent:* Tuesday, August 2, 2016 10:07 PM
> *To:* Paul Cotton <Paul.Cotton@microsoft.com>; Jean-Yves Avenard <
> jya@mozilla.com>; Jerry Smith (WPT) <jdsmith@microsoft.com>
> *Cc:* Chris Pearce <cpearce@mozilla.com>; Matt Wolenetz <
> wolenetz@google.com>; Anthony Jones <ajones@mozilla.com>; Philippe Le
> Hegaret (plh@w3.org) <plh@w3.org>
> *Subject:* RE: Update on MSE and EME Testing
>
>
>
> Sure.
>
>
>
> I believe I have opened some bugs on web-platform-tests GitHub ; but no
> progresses were made.
>
>
>
> Note that we made some modifications in regards to the handling of the
> duration change and that the coded frame removal algorithm can no longer be
> interrupted.
>
> The changes were made in our copy of the tests; it is my understanding
> they will be synced automatically soon.
>
>
>
> The bugs tracking those changes are:
>
> https://bugzilla.mozilla.org/show_bug.cgi?id=1286810 (duration change and
> throwing an error when aborting during a range removal is in progress)
>
> https://bugzilla.mozilla.org/show_bug.cgi?id=1286796 (replacing
> InvalidAccessError with TypeError)
>
>
>
> Kind regards
>
> Jean-Yves
>
>
>
>
>
> Sent from Mail <https://go.microsoft.com/fwlink/?LinkId=550986> for
> Windows 10
>
>
>
> *From: *Paul Cotton <Paul.Cotton@microsoft.com>
> *Sent: *Wednesday, 3 August 2016 9:33 PM
> *To: *Jean-Yves Avenard <jya@mozilla.com>; Jerry Smith (WPT)
> <jdsmith@microsoft.com>
> *Cc: *Chris Pearce <cpearce@mozilla.com>; Matt Wolenetz
> <wolenetz@google.com>; Anthony Jones <ajones@mozilla.com>; Philippe Le
> Hegaret (plh@w3.org) <plh@w3.org>
> *Subject: *RE: Update on MSE and EME Testing
>
>
>
> + Philippe
>
>
>
> Can I please have permission to post this thread to
> public-html-media@w3.org so that these results and possible problems with
> the MSE test suite are broadly known.
>
>
>
> /paulc
>
>
>
> *From:* Jean-Yves Avenard [mailto:jya@mozilla.com <jya@mozilla.com>]
> *Sent:* Wednesday, August 3, 2016 3:55 AM
> *To:* Jerry Smith (WPT) <jdsmith@microsoft.com>
> *Cc:* Chris Pearce <cpearce@mozilla.com>; Matt Wolenetz <
> wolenetz@google.com>; Paul Cotton <Paul.Cotton@microsoft.com>; Anthony
> Jones <ajones@mozilla.com>
> *Subject:* Re: Update on MSE and EME Testing
>
>
>
> Hello Jerry
>
>
>
> On Wed, Aug 3, 2016 at 3:24 AM, Jerry Smith (WPT) <jdsmith@microsoft.com>
> wrote:
>
> Hi Chris and Jean-Yves,
>
>
>
> I previously had some exchanges with you on MSE and EME testing.  The HTML
> Media Extensions group is in the stretch run on both specs, and I have a
> few feature questions to confirm with you.  Matt Wolenetz from Google is
> on this thread as well and may have follow on questions to mine.
>
>
>
> *MSE:*
>
>
>
> There’s an MSE test report here:
> http://w3c.github.io/test-results/media-source/all.html.  It currently
> shows a number of interface tests that seem to be timing out.
>
>
>
> We do see tests intermittently timing out.
>
> You'll find here the expected results and tests skipped for those
> media-source test.
>
> http://hg.mozilla.org/mozilla-central/file/tip/testing/web-platform/meta/media-source
>
> We've worked hard to pass all the media-source webref tests. Thorough
> investigations were performed when we didn't pass some tests, and i can
> pretty confidently say that other than those related to features we do not
> support (e.g. tracks), every tests actually failing or timing out are
> actually a problem in the test.
>
> For the tests disabled, you'll find the bugzilla number on why the test
> was disabled.
>
> Those are typically intermittent timeout or failures.
>
> Unfortunately, the web-platform-test architecture doesn't allow to add a
> comment for the tests we didn't disabled, but instead expected failures
>
>
>
> We have come to the conclusion that those tests often just verify Chrome
> specific behaviour (and verified as chrome being the only browser passing
> those tests).
>
> One example is the buffered range test. The mp4 sample used in all the
> tests do not start at 0. The first PTS is actually 0.096 (only the dts is
> 0). As such, the buffered range start shouldn't be 0.
>
> Many tests actually first check that the buffered range start at 0 and
> abort early, so we get an early failure.
>
> For some intermittent failures or timeout, those are the seek tests or
> playback progress tests. Often those tests fail because they expect media
> events to be received in a particular order. The timeupdate event in
> particular often gets in the way and cause the tests to fail.
>
>
>
> Ignoring that, we still lack two passing implementations on several
> features.  You can see those here:
> http://w3c.github.io/test-results/media-source/less-than-2.html.  The
> feature gaps I previously sent you still show as lacking implementations in
> the latest test results:
>
>
>
>
>
> I'm not sure how the tests are run, but that's certainly not what I'm
> seeing from a nightly build here :(
>
>
> 1.       The on{event} attributes
>
> done
>
> 2.       Live seekable range
>
> done
>
> 3.       audioTracks and videoTracks in Firefox
>
> won't do
>
> 4.       Resetting the delaying-the-load event in Edge (this makes most
> test files timeout but note individual tests may still pass)
>
> done
>
> 5.       TrackDefault (feature marked at-risk)
>
> won't do.
>
>
>
> *Jean-Yves:*  I believe you planned to implement the eventHandler
> attributes and live seekable range.  What build should we be testing to use
> them?  And are audioTracks/videoTracks and TrackDefaults still not planned?
>
>
>
> They will be in our Nightly build after July 16th 2016.
>
>
>
>
> On MSE features identified as at risk in the spec:
>
>
>
> 1.       appendStream is planned to be removed (since it is not current).
>
> 2.       VideoPlaybackQuality is proposed to move into an extension
> spec.  I believe only Microsoft has pushed hard to retain this, and appears
> to be the only browser passing all tests (though only 2 of 20 tests lack
> two passing, both on totalFrameDelay).
>
>
>
> Hmm... we do support VideoPlaybackQuality, I don't believe we support
> totalFrameDelay however.
>
> 3.       The TrackDefault object and its related objects are still TBD.
> (Chrome may have this behind a flag – Matt please confirm).
>
>
>
>
>
> we don't support track related object/attributes. I'm not sure if we have
> any plans to do so.
>
> We do have everything in place to support it, just never given it any
> priority on our tasks list.
>
> Hope this help
>
> Jean-Yves
>
>
>
>
>