Re: Proposal to Advance EME to Proposed Recommendation

On Mon, Oct 24, 2016 at 5:36 PM, Chris Pearce <cpearce@mozilla.com> wrote:

> EME has multiple shipping implementations; it's time it became a standard.
> Mozilla's priority is to see EME v1 finished without needing to go back to
> CR and extend the working group charter.
>
> To that end, we'd prefer to see features without significant
> implementation experience removed from the v1 spec. So we agree with
> Netflix and support removing persistent-license and persistent-usage-record
> removed from the v1 spec. They can be punted to v2.
>

​Just to be clear, whilst we'd also be ok with this, it is not exactly what
I proposed.​

I do think it is hard to distinguish persistent-license and
persistent-usage-record in terms of actual implementation and deployment
experience. If anything, persistent-usage-record has more (at least I am
not aware of significant use of persistent-license in the field, but then
there is no reason that I should be).

However, persistent-license at least has an implementation that passes our
tests without help from JS. If that is part of our criteria, we could go
ahead on that basis with only persistent-usage-record deferred to v2.

...Mark


>
> As for the readyState changes, the behaviour in question was only recently
> added, and the behaviour we'd like to see instead of what's in the spec now
> already falls out of Firefox's and Chrome's existing readyState
> implementations, and this issue is currently being resolved on GitHub.
>
>
> Chris Pearce.
>
>
> On Sat, Oct 22, 2016 at 3:15 PM, Mark Watson <watsonm@netflix.com> wrote:
>
>> All,
>>
>> The proposed implementation experience criteria below do not lead to the
>> conclusion that has been proposed.
>>
>> If this is the path the group would like to take, I propose we do so
>> based on implementation experience criteria that includes *at least one
>> compliant implementation of each feature*, in addition to the criteria
>> proposed by Microsoft. This would be a lesser relaxation of the standard
>> criteria than proposed by Microsoft.
>>
>> Specifically, I do not see how *persistent-license* passes the proposed
>> criteria but *persistent-usage-record* does not: there is at least equal
>> implementation and deployment experience in browsers with
>> *persistent-usage-record* functionality - based on pre-standard APIs -
>> to *persistent-license*. If we are to include such pre-standard
>> implementation and deployment experience in our evaluation, as is proposed
>> for *persistent-license*, we should also do so for
>> *persistent-usage-record*. Just as '“*Persistent-license”* sessions add
>> requirements for saving, re-using and destroying licenses',
>> *persistent-usage-record* sessions add requirements for saving,
>> retrieving and destroying usage information; with the same APIs and flows.
>>
>> Point 2 below states "“*Temporary”* sessions are the primary EME session
>> type that has been broadly tested commercially via user agents and
>> commercial site implementations." But as the largest user of EME, a large
>> fraction of our sessions are *persistent-usage-record* ones (again, based
>> on pre-standard APIs) across three browsers (and at one point four and with
>> one more in test). The underlying functionality is supported by at least
>> three different DRMs.
>>
>> I certainly agree that this kind of experience - with the same
>> functionality and earlier-specification-version APIs - is relevant to the
>> questions of implementability and market demand. But it does not - in the
>> case of either persistent session type - prove out the exact API design we
>> have now: you need a real implementation for that.
>>
>> Where we are lacking with *persistent-usage-record* is only in that
>> fully compliant implementation, now. Since the API for the two persistent
>> sessions is so similar, it's hard to see a difference on any criteria, but
>> at least actual compliant implementations is the traditional W3C criteria
>> and the situation there is clear.
>>
>> ...Mark
>>
>>
>>
>> On Fri, Oct 21, 2016 at 4:21 PM, David Dorwin <ddorwin@google.com> wrote:
>>
>>> We support the path proposed by Microsoft to move forward to Proposed
>>> Recommendation and remove currently at-risk features. We believe it
>>> provides a solid base that demonstrates extensibility, such as through
>>> session types, and unblocks incubation of additional functionality.
>>>
>>> On Fri, Oct 21, 2016 at 11:35 AM, Jerry Smith (WPT) <
>>> jdsmith@microsoft.com> wrote:
>>>
>>>> At Microsoft, we think the EME specification is ready to transition to
>>>> Proposed Recommendation status and do not believe reissuing an EME CR with
>>>> further features “at risk” is necessary. We think the work to develop the
>>>> EME test suite has been extremely valuable and we will continue to use
>>>> these tests when updating our Edge EME implementation. The test suite has
>>>> confirmed where we have features with two passing “independent”
>>>> “implementations” and also provides good evidence of implementation
>>>> experience across most of the EME specification. This leads us to believe
>>>> that we already have sufficient implementation experience
>>>> <http://www.w3.org/2015/Process-20150901/#implementation-experience>
>>>> as required by the W3C Process document.
>>>>
>>>>
>>>>
>>>> Some brief points on our current testing status as of Monday, 10/17:
>>>> We’ve largely completed a test suite of 306 subtests and currently have 2
>>>> independent implementations passing all but 68 tests (78% pass rate).
>>>> 42 of these 68 failures (14% of the full test suite) are related to
>>>> incorrect implementation error handling based on recent specification
>>>> changes that aligned EME with current error handling norms. These
>>>> failures really should be considered as implementation bugs and should not
>>>> block progress of the specification within W3C.  That leaves only 26
>>>> non-passing tests (a 91% pass rate) as of Monday, and test results will
>>>> improve with fixes being checked in this week.  In other words, our
>>>> current test suite results are actually quite good.
>>>>
>>>>
>>>>
>>>> *Our Proposal:*
>>>>
>>>>
>>>>
>>>> We would like to adopt a plan that does NOT republish an EME Candidate
>>>> Recommendation with additional “at risk” features.  To do that, we propose:
>>>>
>>>>
>>>>
>>>> 1.       EME be considered substantially proven by broad
>>>> implementation experience in commercial use on sites like Netflix, YouTube
>>>> and Amazon.  This is permitted via the “judgement level” of the “Model CR
>>>> Exit Criteria (Public Permissive version 3)” referenced in the EME
>>>> Candidate Recommendation.
>>>>
>>>> 2.       The recent experience with the EME test suite supports that
>>>> the EME spec is implementable, most specifically on “*temporary”*
>>>> sessions that exercise most core features of the spec.  “*Temporary”*
>>>> sessions are the primary EME session type that has been broadly tested
>>>> commercially via user agents and commercial site implementations.
>>>>
>>>> 3.       We believe that the “*persistent-license”* sessions feature
>>>> further builds on the base feature set already proven by experience for “
>>>> *temporary”* sessions.  “*Persistent-license”* sessions add
>>>> requirements for saving, re-using and destroying licenses persisted on a
>>>> client device so that they are available for offline playback (e.g. through
>>>> web apps).  Although we don’t yet have *two* passing independent
>>>> implementations, the “*persistent-license”* sessions feature is an
>>>> optional extension to the core EME mechanisms and should be judged to be
>>>> proven by implementation experience to date including experience from*
>>>> “temporary”* sessions.
>>>>
>>>> 4.       This leaves the “at risk” features already identified in the
>>>> current EME Candidate Recommendation:  “*persistent-usage-record”*
>>>> (optional) and “*readyState”* changes when an EME implementation is
>>>> waiting for a key. We do not need to return to Candidate Recommendation to
>>>> remove either of these “at risk” features.
>>>>
>>>> a.       *“Persistent-usage-record”* passed a lengthy W3C TAG review
>>>> process but we have substantially agreed as a group that this feature needs
>>>> additional implementation support to prove it out.  That proof is currently
>>>> lacking and does not appear likely to be available in the EME V1 timeframe
>>>> even if we published a new Candidate Recommendation.  If that is true then
>>>> we believe the “*persistent-usage-record”* feature should be moved out
>>>> of the EME V1 spec.  Depending on continued interest, it may move to a
>>>> separate spec for additional development and/or implementation support.
>>>>
>>>> b.       The *“readyState”* feature is currently being discussed via
>>>> several recent Github issues.  It’s unclear currently whether it should be
>>>> included in EME V1 or not.  We do not believe we should republish a EME CR
>>>> with just this feature marked “at risk”.
>>>>
>>>> A proposed implementation experience criteria consistent with the above
>>>> is:
>>>>
>>>>
>>>>
>>>> 1.       Positive *interoperable* *implementation* experience with at
>>>> least two *independent* browser engines.
>>>>
>>>> 2.       Support or commitment to support *compliant implementations*
>>>> by at least two *independent* browser vendors.
>>>>
>>>> 3.       Positive, *but not necessarily fully passing*, results in our
>>>> test suite by at least one *implementation*.
>>>>
>>>>
>>>>
>>>> *The Take Home Message:*
>>>>
>>>>
>>>>
>>>> We want to propose that EME be published as a Proposed Recommendation
>>>> based on our current level of testing and implementation experience and
>>>> that we remove the “at risk” *persistent-usage-record* feature and
>>>> perhaps the *readyState* interaction portions (depending on
>>>> disposition of the related Github issues).  This would produce a highly
>>>> interoperable EME V1 Proposed Recommendation and the aspects moved out of
>>>> it would not be blocked from proceeding in an EME V2 or in a separate
>>>> specification.
>>>>
>>>>
>>>>
>>>> We’d like to gain agreement on this proposal within the HME Working
>>>> Group and then propose it to the W3C Director as an alternate plan for
>>>> completing the WG’s work on the EME specification.
>>>>
>>>>
>>>>
>>>
>>>
>>
>