Re: Secure release and persistence

Here is a Pull Request in fulfilment of my Action from the F2F:
https://github.com/w3c/encrypted-media/pull/54

You can see the updated specification at
http://mwatson2.github.io/encrypted-media/

I will provide a description of the changes in Issue#45 and update the
use-case wiki with further information as requested.

...Mark

On Thu, Apr 30, 2015 at 2:59 PM, Jerry Smith (WINDOWS) <
jdsmith@microsoft.com> wrote:

>  I think the tone of this email has been relatively civil given some of
> the strong opposing opinions involved.  One thing seems to be clearly
> stated now that maybe wasn’t before:  Mark has confirmed that Netflix has
> no plans to insist on delayed browser close/automatic messaging, and they
> are open to spec language that states this.
>
>
>
> There still appears to be significant concern about the feature
> maintaining its optional status.  I’d be interested in others comments on
> this.  My own take is that the non-real time aspect of the feature make it
> poorly suited to binding features to its presence.   I’ve previously said
> that having it supported on a subset of UAs can still add value, with no
> feature binding involved.  Broad UA support is preferable, but doesn’t seem
> a requirement for the feature to be justified.
>
>
>
> Jerry
>
>
>
> *From:* Mark Watson [mailto:watsonm@netflix.com]
> *Sent:* Thursday, April 30, 2015 11:12 AM
> *To:* David Dorwin
> *Cc:* public-html-media@w3.org
> *Subject:* Re: Secure release and persistence
>
>
>
>
>
>
>
> On Wed, Apr 29, 2015 at 5:37 PM, David Dorwin <ddorwin@google.com> wrote:
>
>  The tone of this thread has turned for the worse. I've addressed some of
> Mark’s comments below, but I'll wait for #1 and #2 before replying further.
>
>
>
> On Wed, Apr 29, 2015 at 9:57 AM, Mark Watson <watsonm@netflix.com> wrote:
>
>
>
>
>
> On Wed, Apr 29, 2015 at 9:29 AM, David Dorwin <ddorwin@google.com> wrote:
>
>  While I welcome further clarification on Netflix's feature, I object to
> further specifying it in the spec for the reasons described in issue 45
> [1], specifically [2].
>
>
>
> ​Well, the F2F agreed to give me an action to provide the remaining
> specification details [4] and I in turn object to the removal of much of
> the existing specification of this feature, which you did last week under
> the guise of a "name change".
>
>
>
> I believe #1 and #2 are consistent with [4]. It doesn't make sense to
> “provide... technical recommendations” for a problem that isn't fully
> defined.
>
>
>
> I’m not sure why you think I removed “much of the existing specification
> of this feature.” Perhaps because there wasn't much actually specified,
> minor deletions appear to be “much” of the specification. I assume you are
> referring to the change to remove() (see below) or [5]. The main changes in
> [5] other than the name were the removal of a) “generate a persistable
> proof of license release”, which was incorrect, and b) specification of
> what the license passed to update() must contain, which depends on further
> definition of the feature. Both were replaced with "*TBD*."
>
>
>
> ​Yes. The feature doesn't need much in the spec and those things
> constituted a major part of it.​
>
>
>
> ​I restored the material in the wiki:
> https://www.w3.org/wiki/HTML/Media_Task_Force/EME_Use_Cases#Limited_Concurrent_Streams_via_Key_Release
>
>
>
> I also made a few updates ​to align with what was in the spec before the
> recent changes.
>
>
>
> Honestly, the level of detail for this feature in the Use-Cases wiki is
> not different from all the other "supported" features. I'd encourage all
> task force members to look at it and tell me if I am going blind somehow.
> I'll happily expand it, but perhaps we should do the same for all the
> features, not just this one ? (see below on this).
>
>
>
> I really don't think it's acceptable for a task force member (editor or
> not) to change a use-case from "Supported" to "Not supported" (or vice
> versa) when that has *not been agreed by the group* and indeed is an
> active topic of discussion. This wiki of use-cases has been stable for a
> while and additions / removals should be by agreement.
>
>
>
>
>
>
>
>  There is still no data justifying the need to add this new requirement
> to all implementations
>
>
>
> ​It's optional.​
>
>
>
> I disagree with this justification. As discussed at the f2f, optional
> features are considered an anti-pattern because they either become required
> to remain competitive or are never implemented. In this case, “optional” is
> de facto required because Netflix (may) require it.
>
>
>
> ​I was just pointing out that "​new requirement to all implementations"
> was not accurate.
>
>
>
> The whole issue of optional features deserves more consideration. This is
> not the only one. It is not obvious, and it's feature-dependent, that an
> optional feature becomes *de facto* required or nowhere implemented. One
> can easily imagine features that enhance user experience where UAs disagree
> on the competitive value vs other things they might implement. Pre-fetching
> of licenses without a <video> element is a good example. So, that it is
> always an "anti-pattern" is your opinion, not something we all agree on.
>
>
>
> Since this issue applies to multiple features, can we leave it here for
> the moment, until we have addressed the other issues ?
>
>
>
>
>
> Regardless, there is no data justifying even the “optional” feature and
> the resulting fragmentation.
>
>
>
>  nor has there been consideration of available alternatives.
>
>
>
> ​License renewal is an alternative which has been extensively discussed,
> has significant dis-advantages and also remains underspecified ​
>
>
>
> I said consideration, not discussion. Specifically, there has been no
> experimentation with the already available feature set. Only your
> assertions that they will not work and Netflix will not consider them.
>
>
>
> ​Secure release is part of the ​"already available feature set" as far as
> I am concerned and we have done more than experiment. It is deployed at
> scale with Safari and IE and it is working well.
>
>
>
> Is license renewal in use on desktop browsers by any service providers ?
> By whom ? Perhaps we can compare notes ?
>
>
>
>
>
>  Both are important prerequisites for adding features to the web
> platform. Furthermore, Netflix's stated use case - whether mentioned in the
> spec or not - requires a fundamental violation of the web platform
> architecture: the CDM/media stack delaying application shutdown.
>
>
>
> ​No, this is not required by our use-case and the more detailed
> specification I will provide this week will make that even clearer, as
> detailed at the top of this thread.​
>
>
>
> Are you saying that with this new solution, Netflix will not require *any*
> user agents to delay shutdown and ensure that messages reach the server?
>
>
>
> ​It's not a "new solution" - it's just more detail on how everyone
> implemented what was there before.
>
>
>
> Delaying shutdown is not mentioned in the use-case wiki, it's not
> mentioned in the specification, we don't require it for our use-case and
> responsive to your concerns I'm *trying* to find ways to make this clearer
> in the spec. If you can find a way for the specification to *prohibit* it,
> that would be fine by me. We don't require any user agents to delay
> shutdown. What else can I do to address your concern ?
>
>
>
>
>
>
>
> Based on these important concerns, clear lack of consensus, continuing
> confusion about this feature even among the editors [3], and lack of data
> or experimentation, it is still too early to include this in the spec and
> require it of all implementations.
>
>
>
> Even with Mark’s description below, there is still confusion about this
> feature [3]. I propose the following next steps for Mark’s feature request:
>
> 1. Mark clearly defines (all) use case(s) for this feature (perhaps in the
> wiki or a doc).
>
> 2. Mark clearly defines the feature he is requesting (perhaps in the wiki
> or a doc).
>
>
>
> ​The use-case and feature definition have been present, first in the
> original specification and later in the wiki, since the beginning of the
> work (i.e. several years). I'm happy to refresh this as I understand you
> may have removed some of this from the wiki recently.
>
>
>
> The use case has been unclear and there has never been a complete feature
> definition. As an example, see the text from “the beginning of the work”:
> [6]. Also, as far as I am aware, this thread is the first time that saving
> decrypt times has been discussed publicly.
>
>
>
> ​The use-case is no less clear than all the others (see below).​
>
>
>
> ​Yes, I'm proposing an additional level of detail based on how everyone
> has implemented this which, since it is common, can be described in the
> specification. It's good for implementation experience to inform the spec.
> I thought it would help with your concerns, since it makes it clear there
> is nothing to be done / persisted / sent etc. at page close.​
>
>
>
>
>
> Yes, I removed the text from the wiki and added a note that the model has
> not been explicitly defined. The text, much of which I had written, was out
> of date, and I did not want it to add confusion to the ongoing discussion.
>
>
>
>  3. Mark provides justification for adding this feature, including data
> on alternatives considered and experimented with as well as user benefit of
> this feature over alternatives.
>
>
>
> ​As I have already explained several times, the user benefit of this
> feature over license renewal is that streams-in-progress do not fail when
> there is a loss of connectivity to the application server or an application
> server outage.​ Put another way, this feature enables detection of
> concurrent stream limit fraud without imposing real-time requirements on
> the application servers. This is both an architectural advantage
> (non-real-time systems always being simpler than real-time ones, on many
> axes) and a user experience benefit.
>
>
>
> It's hard to quantify the benefit and unnecessary to the justification,
> but nevertheless I am working on this.
>
>
>
> So, I agree with steps 1-3 and they have been completed (modulo
> restoration of material you have deleted from the wiki).
>
>
>
> I disagree that steps 1-3 have been completed. Specifically, this thread
> has raised new information. As I said above, I think steps 1-2 are are
> prerequisites for [4].
>
>
>
> The old wiki text is all there in the history. However, I highly doubt
> simply reverting the deletion addresses these steps. We need sufficient
> detail to understand the use cases, implementation requirements,
> algorithms, and whether delayed shutdown might reappear as a requirement on
> some platforms.
>
>
>
> ​I added the following here:
> https://www.w3.org/wiki/HTML/Media_Task_Force/EME_Use_Cases#Failed_Handshake
>
>
>
> "It is not required to attempt the handshake except by the explicit
> procedure described above (for example if the web application is closed or
> crashes)."​
>
>
>
>
>
>  4. The group discusses and attempts to reach consensus on a solution for
> #1.
>
>
>
> ​You are treating this as a brand new feature request wheras I consider it
> a feature that has been present since day one. It is already supported by
> two User Agents. Whilst concerns have been expressed previously, the
> outright opposition is ver​y new. I had expected those earlier concerns to
> be addressed through the interaction between implementation experience and
> specification development. Given the implementation status we are ready to
> complete the specification work. Just because your position changes from
> concerns (which I am addressing in this very thread as well as before) to
> opposition does not set back the feature to a square one requirements
> discussion.
>
>
>
> The definition you appear to be proposing is brand new
>
> ​.​
>
>  Earlier this month, you were arguing for a different solution. If some of
> this information is new to me and the public discussion, I find it hard to
> believe “It is already supported by two User Agents.”
>
>
>
> ​Well, it is indeed supported and deployed. You can look at the API
> interactions between our JS on netflix.com and the browsers in question
> if you find it hard to believe.
>
>
>
> As noted above, I'm just proposing additional detail about how the feature
> is implemented under the covers. Whilst previous text talks about
> persisting the license release message, it is always the case that ​you can
> implement however you like so long as the API behavior is the same. So,
> it's just a different way of describing the same thing - actually with a
> bit more detail. We can go back to the previous way if you like, but I do
> think this way is better.
>
>
>
>   Not to mention, there is no clear definition of what “it” is. The spec
> process is more than just documenting existing implementations. Features
> aren’t typically dictated by a single site, either. Perhaps there are
> others that would like similar functionality, but they would be excluded by
> the rubber-stamping of such privately-developed definitions.
>
>
>
> ​Well, sites are restricted to using the DRMs supported by the User
> Agents​. Where these DRMs already support a feature in a common way, I
> think it makes sense for us to support that in our specification.
>
>
>
>
>
> The spec started as a proposal for some simple APIs to perform license
> exchanges. Through the spec process, including input from WG members and
> the TAG, the importance of interoperability, fragmentation,
> implementability, privacy, security, etc. have all become clearer. We've
> also learned of additional use cases and discussed various user agent
> implementations. Through this process, the bar for features and
> functionality has been raised (to be consistent with the general spec
> process). Having never been concretely defined and posing a number of
> concerns, it seems reasonable for this feature to be expected to reach this
> bar.
>
>
>
> ​I'm glad you bring up this evolution in our process which has largely
> been driven in this group by you. I agree with the direction and I think it
> would help if we could expand that to explicit agreement of the whole
> group. We should discuss in more detail what this means for our work,
> because I think you are making assumptions that are not shared by the group
> (at least, not yet).
>
>
>
> I'm more than happy to expand the use-case, definition and specification
> of the feature to meet a higher bar. We should do the same for other
> features. I think we should just get to work collaborating on that, rather
> than ripping (certain) things out and starting from scratch. I think that
> was the conclusion of the F2F.
>
>
>
>
>
> We have opposed this feature “since the beginning of the work.” At one
> point, it was "allowed" due to underspecification. Five months ago, a
> session type was added because we needed to specify behavior for a
> different use case ("persistent-license"). A month ago, I accidentally
> specified the remove() behavior was the same for both session types.
>
>
>
> ​I have seen this problem many times in standardization: you add something
> to the specification "by mistake" which others think was deliberate and
> agree with. People read it, even implement based on that. Later you realize
> your mistake but at this point it is too late. You need to discuss the
> reversion, why it was a mistake, consider that perhaps even if it was
> unintentional, it was not *actually* a mistake for the specification,
> since it passed group review etc.
>
>
>
>
>
> We have always strongly objected to any scenario that involved delaying
> shutdown. There was a period of time where we were led to believe various
> things were no longer required, at which time our opposition was lessened.
> However, that turned out to be false and/or subject to change, and our
> opposition returned. You'll have to forgive me if I'm cautious about any
> new claims that delaying shutdown is off the table.
>
>
>
> ​Fair enough. See above.​
>
>
>
>
>
> ​...Mark​
>
>
>
>
>
>
>
> It’s worth noting that without such evolution to the group's approach to
> the spec and our consistent pushback, delayed shutdown might well be a de
> facto requirement for all implementations, including desktop browsers. If
> we are now at a point (different from even earlier this month) where that
> is no longer being considered, then I would call that a success for the web
> platform.
>
>
>
>
>
> ...Mark
>
>
>
>
>
> [1] https://github.com/w3c/encrypted-media/issues/45
>
> [2] https://github.com/w3c/encrypted-media/issues/45#issuecomment-91743387
>
> [3]
> https://lists.w3.org/Archives/Public/public-html-media/2015Apr/0067.html
>
>
>
> ​[4] http://www.w3.org/2015/04/16-html-media-minutes.html#action01]​
>
>
>
>
>
> David
>
>
>
> P.S.
>
> "Record of license destruction" was only intended for “persistent-license”,
>
>
>
> ​Whatever your intention, what was actually in the specification until
> your recent changes applied this to both ​“persistent-license” and
> “persistent-release”.
>
>
>
> There was one inadvertent case where this was the case. It was added a
> month ago in the commit to ‘Define the behavior for "persistent-license"
> sessions’ [7]. In your original email, you said it was incorrect for your
> use case. Fixing this mistake and separating the session types still seems
> like the appropriate action.
>
>
>
>  but it is similarly restrictive for implementations. I’ll change it to
> something like "persist an invalidation of the license." Other wording
> suggestions welcome.
>
>
>
>  [5]
> https://github.com/w3c/encrypted-media/commit/71e4352d4a737f3cdba879217dc9daf74020751d
>
> [6]
> https://dvcs.w3.org/hg/html-media/raw-file/eme-v0.1/encrypted-media/encrypted-media.html#key-release
>
> [7]
> https://github.com/w3c/encrypted-media/commit/76c48b4f0a171af89ed93c21745900ecbb75f7a2
>
>
>
>
>
>
>
>
> On Tue, Apr 28, 2015 at 8:02 AM, Mark Watson <watsonm@netflix.com> wrote:
>
>  All,
>
>
>
> Hopefully this will catch you in your newly-freed "EME hour" :-)
>
>
>
> As promised at the F2F I will draft spec updates this week to fill in the
> missing details of this feature. However, I would like to make one
> significant change in response to comments at the F2F and previously
> regarding exactly what is persisted.
>
>
>
> The spec describes persistence of a "record of license destruction". Aside
> from the fact that we have no concept in our spec of "license", this is not
> typically how this feature is implemented in DRMs and suggests a need to
> persist data at close / shutdown / crash or other events that cause license
> destruction.
>
>
>
> In practice, what is persisted  is a record of available keys at a point
> in time. This is updated regularly *during streaming*. There is no update
> on close / shutdown / crash. In a later browsing session, this record is
> compared with the actually available keys and a discrepancy is taken as
> evidence that keys were destroyed.
>
>
>
> I propose that we describe this persisted data explicitly as "for each key
> in the session, the *first decrypt time* and the *latest decrypt time*".*
>
>
>
> Are there any comments on that before I implement it ?
>
>
>
> Based on this change in description, I suggest we call this session type "
> *tracked*" - or something similar - since really we are 'tracking' the
> usage of keys. This name will also invite appropriate scrutiny of the
> persisted data properties.
>
>
>
> ...Mark
>
>
>
> * there may of course be other CDM-specific information persisted and
> included in the release message. For example if the CDM has a concept of
> licenses, then license correlation identifiers may be present. Also, how
> the CDM communicates time in its messages may be CDM-specific.
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>