- From: David Dorwin <ddorwin@google.com>
- Date: Fri, 28 Oct 2016 09:10:01 -0700
- To: Chris Pearce <cpearce@mozilla.com>
- Cc: "Jerry Smith (WPT)" <jdsmith@microsoft.com>, Paul Cotton <Paul.Cotton@microsoft.com>, Mark Watson <watsonm@netflix.com>, Matt Wolenetz <wolenetz@google.com>, "public-html-media@w3.org" <public-html-media@w3.org>
- Message-ID: <CAHD2rsiETY-D3pUX5+xgOwzsm_pD3G2Qhep9DA0CZvH+AgQK2A@mail.gmail.com>
Thanks, Jerry. The CDM change Chris mentioned for the drm-mp4-playback-temporary-events.html failure is tracked in https://crbug.com/659828. The drm-mp4-playback-persistent-license-events.html failure on Chrome OS is tracked as https://crbug.com/660393. drm-mp4-playback-temporary-expired.html is flaky due to timing. I just saw a failure and a pass. drm-mp4-unique-origin.html appears to be a new failure. We're looking into it. On Thu, Oct 27, 2016 at 6:43 PM, Chris Pearce <cpearce@mozilla.com> wrote: > The failure in /encrypted-media/drm-mp4-playback-temporary-events.html > (at least in Chrome and in Firefox) is caused by Widevine CDM resolving > MediaKeySession.closed and the promise returned by MediaKeySession.close() > in the wrong order. > > The test changed with issue 4058 > <https://github.com/w3c/web-platform-tests/issues/4058> to expect that > the MediaKeySession.closed promise is resolved before the promise returned > by MediaKeySession.close() is resolved. Both Firefox and Chrome depend on > the Widevine CDM to resolve the promise in the correct order, and it > doesn't. So this is an implementation bug in the Widevine CDM. > > The ClearKey variant of this test > <https://w3c-test.org/encrypted-media/clearkey-mp4-playback-temporary-events.html> > should be fixed in Firefox Nightly in a day or so, and is already fixed in > the latest Chrome Canary. > > We have patches in the works for fixing drm-mp4-playback-temporary- > multikey-sequential[-readyState].html. > > Chris. > > > > On Fri, Oct 28, 2016 at 1:28 PM, Jerry Smith (WPT) <jdsmith@microsoft.com> > wrote: > >> Summary of issues related to each failure is: >> >> >> >> Persistent-license failures - 1 Failures >> >> /encrypted-media/drm-mp4-playback-persistent-license-events.html (1/1, >> 100.00%, 0.33% of total) >> >> drm, persistent-license, mp4, playback, check events >> >> >> >> This is being internally investigated at Google for a Chrome OS fix. >> >> >> >> Temporary session failures - 2 Failures >> >> /encrypted-media/drm-temporary-license-type.html (1/1, 100.00%, 0.33% of >> total) >> >> drm, cannot load persistent license into temporary session >> >> >> >> This is being worked under https://github.com/w3c/web-platform-tests/issues/4027 >> assigned to mwatson2 >> >> >> >> /encrypted-media/drm-mp4-playback-temporary-events.html (1/1, 100.00%, >> 0.33% of total) >> >> drm, temporary, mp4, playback, check events >> >> >> >> This is actually a new failure that didn’t occur on the 10/24/2016 >> pass. Can anyone comment on the test? >> >> >> >> Temporary session readyState failures (listed at risk, under discussion) >> - 2 Failures >> >> /encrypted-media/drm-mp4-playback-temporary-multikey-sequential-readyState.html >> (1/1, 100.00%, 0.33% of total) >> >> drm, successful playback, temporary, mp4, multiple keys, sequential, >> readyState >> >> /encrypted-media/drm-mp4-playback-temporary-multikey-sequential.html >> (1/1, 100.00%, 0.33% of total) >> >> drm, successful playback, temporary, mp4, multiple keys, sequential >> >> >> >> This issue is dependent on some spec clarifications in these issues: >> >> https://github.com/w3c/encrypted-media/issues/338 - closed >> >> https://github.com/w3c/encrypted-media/issues/339 - closed >> >> https://github.com/w3c/encrypted-media/issues/336 - in review >> >> >> >> I’m not sure if we have an issue opened to modify the test once the >> spec issues are cleared. >> >> >> >> Jerry >> >> >> >> *From:* Jerry Smith (WPT) [mailto:jdsmith@microsoft.com] >> *Sent:* Thursday, October 27, 2016 4:40 PM >> *To:* Paul Cotton <Paul.Cotton@microsoft.com>; David Dorwin < >> ddorwin@google.com>; Mark Watson <watsonm@netflix.com>; Matt Wolenetz < >> wolenetz@google.com>; 'public-html-media@w3.org' < >> public-html-media@w3.org> >> *Subject:* EME Test Status Update - 10-27-16 >> >> >> >> I’ve posted updated test status reports to reflect a few bug fixes and >> also to add Chrome OS results. >> >> >> >> Reports: >> >> >> >> · http://w3c.github.io/test-results/encrypted-media/all.html >> >> o *Test files: 97; Total subtests: 306* >> >> · http://w3c.github.io/test-results/encrypted-media/complete- >> fails.html >> >> o *Completely failed files: 16; Completely failed subtests: 16; >> Failure level: 16/306 (5.23%)* >> >> · http://w3c.github.io/test-results/encrypted-media/less-than- >> 2.html >> >> o *Test files without 2 passes: 28; Subtests without 2 passes: 40; >> Failure level: 40/306 (13.07%)* >> >> Comments: >> >> >> >> - Complete-fails dropped from 21 to 16, and Less-Than-2 dropped from >> 47 to 40 subtest failures. >> >> - Persistent-license failures dropped from 5 to 1 with the addition of >> Chrome on Chrome OS results. >> >> - One temporary session test failed that was previously passing: >> >> o /encrypted-media/drm-mp4-playback-temporary-events.html (1/1, >> 100.00%, 0.33% of total) >> >> o drm, temporary, mp4, playback, check events >> >> - There are only 5 tests in complete-fails that we are still working >> on for our conformance record. >> >> - That reflects our revised criteria of having a single passing >> implementation (in addition to broad interoperable experience). >> >> In line with that criteria, I’ve chosen to break down just the >> complete-fails test results for this summary. >> >> >> >> Browser Versions: >> >> >> >> CD56: Chrome 56.0.2902.0 canary (64-bit) >> >> CO56: Chrome OS 56.0.2900.0 canary (64-bit) >> >> ED14: Edge 14, Windows 10.0.14393.351 (64-bit) >> >> FF52: Firefox 52.0a1 (2016-10-27) (32-bit) >> >> >> >> Summary Groupings; >> >> >> >> Errors (considered bugs, not blocking) >> 0 >> >> Clear Key persistent-license (not planned, may make >> non-normative) 5 >> >> Clear Key persistent-usage-record (at risk, planned to be >> removed) 3 >> >> Clear Key transitions between clear and encrypted (bugs, not >> blocking) 0 >> >> >> >> Not Blocking Tests >> 8 >> >> >> >> Persistent-license (fixes in work) >> 1 >> >> Persistent-usage-record (at risk, planned to be >> removed) 3 >> >> Temporary session edge cases (bugs, not block, fixes in >> work) 2 >> >> Temporary session readyState (issues under discussion, at risk) >> 2 >> >> >> >> In work or at risk >> 8 >> >> >> >> TOTAL >> 16 >> >> >> >> >> >> Details on failed tests are below. >> >> >> >> Jerry >> >> >> >> >> >> Error handling (considered bugs, not blocking) - 0 Failures >> >> >> >> Clear key persistent-license tests (not planned to be implemented, may >> make informative note) - 5 Failures >> >> /encrypted-media/clearkey-mp4-playback-destroy-persistent-license.html >> (1/1, 100.00%, 0.33% of total) >> >> org.w3.clearkey, persistent-license, mp4, playback, destroy and >> acknowledge >> >> /encrypted-media/clearkey-mp4-playback-persistent-license-events.html >> (1/1, 100.00%, 0.33% of total) >> >> org.w3.clearkey, persistent-license, mp4, playback, check events >> >> /encrypted-media/clearkey-mp4-playback-persistent-license.html (1/1, >> 100.00%, 0.33% of total) >> >> org.w3.clearkey, persistent-license, mp4playback >> >> /encrypted-media/clearkey-mp4-playback-retrieve-destroy-persistent-license.html >> (1/1, 100.00%, 0.33% of total) >> >> org.w3.clearkey, persistent-license, mp4, playback, retrieve, playback >> and destroy >> >> /encrypted-media/clearkey-mp4-playback-retrieve-persistent-license.html >> (1/1, 100.00%, 0.33% of total) >> >> org.w3.clearkey, persistent-license, mp4, playback, retrieve and playback >> >> >> >> Clear key persistent-usage-record (not planned to be implemented) - 3 >> Failures >> >> /encrypted-media/clearkey-mp4-playback-persistent-usage-record-events.html >> (1/1, 100.00%, 0.33% of total) >> >> org.w3.clearkey, persistent-usage-record, mp4, playback, check events >> >> /encrypted-media/clearkey-mp4-playback-persistent-usage-record.html >> (1/1, 100.00%, 0.33% of total) >> >> org.w3.clearkey, persistent-usage-record, mp4playback >> >> /encrypted-media/clearkey-mp4-playback-retrieve-persistent-usage-record.html >> (1/1, 100.00%, 0.33% of total) >> >> org.w3.clearkey, persistent-usage-record, mp4, playback, retrieve in new >> window >> >> >> >> Clear key transitions between clear and encrypted content (considered >> bugs, not blocking) - 2 Failures >> >> >> >> Persistent-license failures - 6 Failures >> >> /encrypted-media/drm-mp4-playback-persistent-license-events.html (1/1, >> 100.00%, 0.33% of total) >> >> drm, persistent-license, mp4, playback, check events >> >> >> >> Persistent-usage-record failures - 3 Failures >> >> /encrypted-media/drm-mp4-playback-persistent-usage-record-events.html >> (1/1, 100.00%, 0.33% of total) >> >> drm, persistent-usage-record, mp4, playback, check events >> >> /encrypted-media/drm-mp4-playback-retrieve-persistent-usage-record.html >> (1/1, 100.00%, 0.33% of total) >> >> drm, persistent-usage-record, mp4, playback, retrieve in new window >> >> /encrypted-media/drm-mp4-playback-persistent-usage-record.html (1/1, >> 100.00%, 0.33% of total) >> >> drm, persistent-usage-record, mp4playback >> >> >> >> Temporary session failures - 2 Failures >> >> /encrypted-media/drm-temporary-license-type.html (1/1, 100.00%, 0.33% of >> total) >> >> drm, cannot load persistent license into temporary session >> >> /encrypted-media/drm-mp4-playback-temporary-events.html (1/1, 100.00%, >> 0.33% of total) >> >> drm, temporary, mp4, playback, check events >> >> >> >> Temporary session readyState failures (listed at risk, under discussion) >> - 2 Failures >> >> /encrypted-media/drm-mp4-playback-temporary-multikey-sequential-readyState.html >> (1/1, 100.00%, 0.33% of total) >> >> drm, successful playback, temporary, mp4, multiple keys, sequential, >> readyState >> >> /encrypted-media/drm-mp4-playback-temporary-multikey-sequential.html >> (1/1, 100.00%, 0.33% of total) >> >> drm, successful playback, temporary, mp4, multiple keys, sequential >> > >
Received on Friday, 28 October 2016 16:10:55 UTC