- From: Philippe Le Hégaret <plh@w3.org>
- Date: Wed, 9 Nov 2016 15:32:10 -0500
- To: "Jerry Smith (WPT)" <jdsmith@microsoft.com>, Paul Cotton <Paul.Cotton@microsoft.com>, "public-html-media@w3.org" <public-html-media@w3.org>
Hi Jerry, I see that we have test results involving polyfills. Do we explain those anywhere? Some of them are about persistent-usage-records so probably no longer relevant here. I'd like to make sure I represent those as accurately as possible. Philippe On 11/2/2016 7:15 PM, Jerry Smith (WPT) wrote: > I’ve posted updated test status reports to EME, with two notable outcomes: > > · Only 3 tests have failures in areas that we consider significant > for spec compliance. I’ve listed proposed plans for all three below. > At least two recommend disabling the existing test. > > o If the plans for these three test failures are adopted, then > there would be no outstanding test to complete for submission. > > · */readyState/*tests both passed on Firefox, which should mean > that at-risk feature is considered as having a compliant implementation. > > I’ve summarized tests with no passing implementation in this report, and > for completeness (not compliance), have attached 3 FF specific test > failures raised by Chris Pearce in an email this week. We have a > passing implementation on each of the 3 tests, but there are possible > corrective actions in work for each. They are listed at the bottom of > the testing summary. > > > > *_Reports:_* > > · http://w3c.github.io/test-results/encrypted-media/all.html > > o *Test files*: 97; *Total subtests*: 306 > > · > http://w3c.github.io/test-results/encrypted-media/complete-fails.html > > o *Completely failed files*: 14; *Completely failed subtests*: > 14; *Failure level*: 14/306 (4.58%) > > · http://w3c.github.io/test-results/encrypted-media/less-than-2.html > > o *Test files without 2 passes*: 28; *Subtests without 2 passes: > *40; *Failure level*: 40/306 (13.07%) > > * * > > *PRIORITY TESTS WITHOUT ONE PASSING IMPLEMENTATION (3 TESTS): * > > *_Persistent-license failures:_* > > /encrypted-media/drm-mp4-playback-persistent-license-events.html (1/1, > 100.00%, 0.33% of total) OK > > drm, persistent-license, mp4, playback, check events FAIL > > Status: Will require pushing a new CDM. > > */ Plan: Consider a bug with commitment to fix > (https://crbug.com/660393) /* > > *_Temporary session failures:_* > > /encrypted-media/drm-temporary-license-type.html (1/1, 100.00%, 0.33% of > total) OK > > drm, cannot load persistent license into temporary session FAIL > > Status: Widevine license server converts license request to > temporary. > > */ Plan: Disable this test for V1. /* > > /encrypted-media/drm-mp4-playback-temporary-events.html (1/1, 100.00%, > 0.33% of total) OK > > drm, temporary, mp4, playback, check events FAIL > > Status: Will require pushing a new CDM. > > */ Plan: Consider a bug with commitment to fix > (https://crbug.com/660393) /* > > > > *TEST DETAILS: * > > *_Browser Versions:_* > > CD56: Chrome 56.0.2907.0 canary (64-bit) > > CO56: Chrome OS 56.0.2901.0 canary (64-bit) > > ED14: Edge 14, Windows 10.0.14393.351 (64-bit) > > FF52: Firefox 52.0a1 (2016-11-01) (32-bit) > > > > *_Failure Summary:_* > > ================================================================================ > > Errors (considered bugs, not > blocking) 0 > > Clear Key persistent-license (not planned, may make > non-normative) 5 > > Clear Key persistent-usage-record (at risk, planned to be > removed) 3 > > Clear Key transitions between clear and encrypted (bugs, not blocking) > 0 > > Persistent-usage-record (at risk, planned to be removed) > 3 > > Clear Key transitions between clear and encrypted (bugs, not > blocking) 0 > > ================================================================================ > > */Total Not Blocking Tests > 11/* > > ================================================================================ > > Persistent-license (fixes in work) > 1 > > Temporary session edge cases (bugs, not block, fixes in work) > 2 > > Temporary session readyState (resolved, no longer at risk) > 0 > > ================================================================================= > > */Total Compliance Related Tests > 3/* > > *TOTAL > 14* > > * * > > *_Failed Tests by Group (14): _* > > *Error handling (considered bugs, not blocking) - 0* > > *Clear key persistent-license tests (not planned to be implemented, may > make informative note) - 5* > > /encrypted-media/clearkey-mp4-playback-destroy-persistent-license.html > (1/1, 100.00%, 0.33% of total) > > org.w3.clearkey, persistent-license, mp4, playback, destroy and > acknowledge > > /encrypted-media/clearkey-mp4-playback-persistent-license-events.html > (1/1, 100.00%, 0.33% of total) > > org.w3.clearkey, persistent-license, mp4, playback, check events > > /encrypted-media/clearkey-mp4-playback-persistent-license.html (1/1, > 100.00%, 0.33% of total) > > org.w3.clearkey, persistent-license, mp4playback > > /encrypted-media/clearkey-mp4-playback-retrieve-destroy-persistent-license.html > (1/1, 100.00%, 0.33% of total) > > org.w3.clearkey, persistent-license, mp4, playback, retrieve, > playback and destroy > > /encrypted-media/clearkey-mp4-playback-retrieve-persistent-license.html > (1/1, 100.00%, 0.33% of total) > > org.w3.clearkey, persistent-license, mp4, playback, retrieve and > playback > > *Clear key persistent-usage-record (not planned to be implemented) - 3* > > /encrypted-media/clearkey-mp4-playback-persistent-usage-record-events.html > (1/1, 100.00%, 0.33% of total) > > org.w3.clearkey, persistent-usage-record, mp4, playback, check events > > /encrypted-media/clearkey-mp4-playback-persistent-usage-record.html > (1/1, 100.00%, 0.33% of total) > > org.w3.clearkey, persistent-usage-record, mp4playback > > /encrypted-media/clearkey-mp4-playback-retrieve-persistent-usage-record.html > (1/1, 100.00%, 0.33% of total) > > org.w3.clearkey, persistent-usage-record, mp4, playback, retrieve in > new window > > *Clear key transitions between clear and encrypted content (considered > bugs, not blocking) - 0* > > *Persistent-license failures - 1* > > /encrypted-media/drm-mp4-playback-persistent-license-events.html (1/1, > 100.00%, 0.33% of total) > > drm, persistent-license, mp4, playback, check events > > *Temporary session failures - 2* > > /encrypted-media/drm-temporary-license-type.html (1/1, 100.00%, 0.33% of > total) > > drm, cannot load persistent license into temporary session > > /encrypted-media/drm-mp4-playback-temporary-events.html (1/1, 100.00%, > 0.33% of total) > > drm, temporary, mp4, playback, check events > > *Temporary session readyState failures (listed at risk, under > discussion) – 0* > > *================================================================================* > > *_Other FF Test Failures from Nov 1 email from Chris Pearce:_* > > /encrypted-media/clearkey-mp4-unique-origin.html & > /encrypted-media/drm-mp4-unique-origin.html > > Status: Tests Chrome implementation, not the spec. > > / *Plan: Disable the test for V1.*/ > > /encrypted-media/drm-events.html > > Status: Test isn't completing in 10 s due to network latency. > > */ Plan: Increase the latency./* > > /encrypted-media/drm-mp4-playback-temporary-expired.html > > Status: Unknown issue. > > */ Plan: Consider a bug with commitment to fix? If so, can there be > a public bug link?/* > > >
Received on Wednesday, 9 November 2016 20:32:20 UTC