Re: Secure release and persistence

I added some additional questions/clarification to the use case (https://www.w3.org/wiki/HTML/Media_Task_Force/EME_Use_Cases#Failed_Handshake <https://www.w3.org/wiki/HTML/Media_Task_Force/EME_Use_Cases#Failed_Handshake>).
Please double-check that this makes sense and follows your expectations.

> On May 1, 2015, at 7:38 AM, Mark Watson <watsonm@netflix.com> wrote:
> 
> David,
> 
> Regarding "delayed shutdown", whilst I think we are on the same page, I think it would be good if you could define more precisely the behaviors you would like to avoid and the architectural principle you would like to see maintained. I don't want us to miss any differences in understanding now - either between us or across browsers - that might cause trouble further down the road.
> 
> Thanks,
> 
> ...Mark
> 
> 
> 
> On Thu, Apr 30, 2015 at 3:29 PM, Mark Watson <watsonm@netflix.com <mailto:watsonm@netflix.com>> wrote:
> Here is a Pull Request in fulfilment of my Action from the F2F: https://github.com/w3c/encrypted-media/pull/54 <https://github.com/w3c/encrypted-media/pull/54>
> 
> You can see the updated specification at http://mwatson2.github.io/encrypted-media/ <http://mwatson2.github.io/encrypted-media/>
> 
> I will provide a description of the changes in Issue#45 and update the use-case wiki with further information as requested.
> 
> ...Mark
> 
> On Thu, Apr 30, 2015 at 2:59 PM, Jerry Smith (WINDOWS) <jdsmith@microsoft.com <mailto:jdsmith@microsoft.com>> wrote:
> I think the tone of this email has been relatively civil given some of the strong opposing opinions involved.  One thing seems to be clearly stated now that maybe wasn’t before:  Mark has confirmed that Netflix has no plans to insist on delayed browser close/automatic messaging, and they are open to spec language that states this.
> 
> 
> 
> There still appears to be significant concern about the feature maintaining its optional status.  I’d be interested in others comments on this.  My own take is that the non-real time aspect of the feature make it poorly suited to binding features to its presence.   I’ve previously said that having it supported on a subset of UAs can still add value, with no feature binding involved.  Broad UA support is preferable, but doesn’t seem a requirement for the feature to be justified.
> 
> 
> 
> Jerry
> 
> 
> 
> From: Mark Watson [mailto:watsonm@netflix.com <mailto:watsonm@netflix.com>]
> Sent: Thursday, April 30, 2015 11:12 AM
> To: David Dorwin
> Cc: public-html-media@w3.org <mailto:public-html-media@w3.org>
> Subject: Re: Secure release and persistence
> 
> 
> 
> 
> 
> 
> 
> On Wed, Apr 29, 2015 at 5:37 PM, David Dorwin <ddorwin@google.com <mailto:ddorwin@google.com>> wrote:
> 
> The tone of this thread has turned for the worse. I've addressed some of Mark’s comments below, but I'll wait for #1 and #2 before replying further.
> 
> 
> 
> On Wed, Apr 29, 2015 at 9:57 AM, Mark Watson <watsonm@netflix.com <mailto:watsonm@netflix.com>> wrote:
> 
> 
> 
> 
> 
> On Wed, Apr 29, 2015 at 9:29 AM, David Dorwin <ddorwin@google.com <mailto:ddorwin@google.com>> wrote:
> 
> While I welcome further clarification on Netflix's feature, I object to further specifying it in the spec for the reasons described in issue 45 [1], specifically [2].
> 
> 
> 
> ​Well, the F2F agreed to give me an action to provide the remaining specification details [4] and I in turn object to the removal of much of the existing specification of this feature, which you did last week under the guise of a "name change".
> 
> 
> 
> I believe #1 and #2 are consistent with [4]. It doesn't make sense to “provide... technical recommendations” for a problem that isn't fully defined.
> 
> 
> 
> I’m not sure why you think I removed “much of the existing specification of this feature.” Perhaps because there wasn't much actually specified, minor deletions appear to be “much” of the specification. I assume you are referring to the change to remove() (see below) or [5]. The main changes in [5] other than the name were the removal of a) “generate a persistable proof of license release”, which was incorrect, and b) specification of what the license passed to update() must contain, which depends on further definition of the feature. Both were replaced with "TBD."
> 
> 
> 
> ​Yes. The feature doesn't need much in the spec and those things constituted a major part of it.​
> 
> 
> 
> ​I restored the material in the wiki: https://www.w3.org/wiki/HTML/Media_Task_Force/EME_Use_Cases#Limited_Concurrent_Streams_via_Key_Release <https://www.w3.org/wiki/HTML/Media_Task_Force/EME_Use_Cases#Limited_Concurrent_Streams_via_Key_Release>
> 
> 
> I also made a few updates ​to align with what was in the spec before the recent changes.
> 
> 
> 
> Honestly, the level of detail for this feature in the Use-Cases wiki is not different from all the other "supported" features. I'd encourage all task force members to look at it and tell me if I am going blind somehow. I'll happily expand it, but perhaps we should do the same for all the features, not just this one ? (see below on this).
> 
> 
> 
> I really don't think it's acceptable for a task force member (editor or not) to change a use-case from "Supported" to "Not supported" (or vice versa) when that has not been agreed by the group and indeed is an active topic of discussion. This wiki of use-cases has been stable for a while and additions / removals should be by agreement.
> 
> 
> 
> 
> 
> 
> 
> There is still no data justifying the need to add this new requirement to all implementations
> 
> 
> 
> ​It's optional.​
> 
> 
> 
> I disagree with this justification. As discussed at the f2f, optional features are considered an anti-pattern because they either become required to remain competitive or are never implemented. In this case, “optional” is de facto required because Netflix (may) require it.
> 
> 
> 
> ​I was just pointing out that "​new requirement to all implementations" was not accurate.
> 
> 
> 
> The whole issue of optional features deserves more consideration. This is not the only one. It is not obvious, and it's feature-dependent, that an optional feature becomes de facto required or nowhere implemented. One can easily imagine features that enhance user experience where UAs disagree on the competitive value vs other things they might implement. Pre-fetching of licenses without a <video> element is a good example. So, that it is always an "anti-pattern" is your opinion, not something we all agree on.
> 
> 
> 
> Since this issue applies to multiple features, can we leave it here for the moment, until we have addressed the other issues ?
> 
> 
> 
> 
> 
> Regardless, there is no data justifying even the “optional” feature and the resulting fragmentation.
> 
> 
> 
> nor has there been consideration of available alternatives.
> 
> 
> 
> ​License renewal is an alternative which has been extensively discussed, has significant dis-advantages and also remains underspecified ​
> 
> 
> 
> I said consideration, not discussion. Specifically, there has been no experimentation with the already available feature set. Only your assertions that they will not work and Netflix will not consider them.
> 
> 
> 
> ​Secure release is part of the ​"already available feature set" as far as I am concerned and we have done more than experiment. It is deployed at scale with Safari and IE and it is working well.
> 
> 
> 
> Is license renewal in use on desktop browsers by any service providers ? By whom ? Perhaps we can compare notes ?
> 
> 
> 
> 
> 
> Both are important prerequisites for adding features to the web platform. Furthermore, Netflix's stated use case - whether mentioned in the spec or not - requires a fundamental violation of the web platform architecture: the CDM/media stack delaying application shutdown.
> 
> 
> 
> ​No, this is not required by our use-case and the more detailed specification I will provide this week will make that even clearer, as detailed at the top of this thread.​
> 
> 
> 
> Are you saying that with this new solution, Netflix will not require *any* user agents to delay shutdown and ensure that messages reach the server?
> 
> 
> 
> ​It's not a "new solution" - it's just more detail on how everyone implemented what was there before.
> 
> 
> 
> Delaying shutdown is not mentioned in the use-case wiki, it's not mentioned in the specification, we don't require it for our use-case and responsive to your concerns I'm *trying* to find ways to make this clearer in the spec. If you can find a way for the specification to *prohibit* it, that would be fine by me. We don't require any user agents to delay shutdown. What else can I do to address your concern ?
> 
> 
> 
> 
> 
> 
> 
> Based on these important concerns, clear lack of consensus, continuing confusion about this feature even among the editors [3], and lack of data or experimentation, it is still too early to include this in the spec and require it of all implementations.
> 
> 
> 
> Even with Mark’s description below, there is still confusion about this feature [3]. I propose the following next steps for Mark’s feature request:
> 
> 1. Mark clearly defines (all) use case(s) for this feature (perhaps in the wiki or a doc).
> 
> 2. Mark clearly defines the feature he is requesting (perhaps in the wiki or a doc).
> 
> 
> 
> ​The use-case and feature definition have been present, first in the original specification and later in the wiki, since the beginning of the work (i.e. several years). I'm happy to refresh this as I understand you may have removed some of this from the wiki recently.
> 
> 
> 
> The use case has been unclear and there has never been a complete feature definition. As an example, see the text from “the beginning of the work”: [6]. Also, as far as I am aware, this thread is the first time that saving decrypt times has been discussed publicly.
> 
> 
> 
> ​The use-case is no less clear than all the others (see below).​
> 
> 
> 
> ​Yes, I'm proposing an additional level of detail based on how everyone has implemented this which, since it is common, can be described in the specification. It's good for implementation experience to inform the spec. I thought it would help with your concerns, since it makes it clear there is nothing to be done / persisted / sent etc. at page close.​
> 
> 
> 
> 
> 
> Yes, I removed the text from the wiki and added a note that the model has not been explicitly defined. The text, much of which I had written, was out of date, and I did not want it to add confusion to the ongoing discussion.
> 
> 
> 
> 3. Mark provides justification for adding this feature, including data on alternatives considered and experimented with as well as user benefit of this feature over alternatives.
> 
> 
> 
> ​As I have already explained several times, the user benefit of this feature over license renewal is that streams-in-progress do not fail when there is a loss of connectivity to the application server or an application server outage.​ Put another way, this feature enables detection of concurrent stream limit fraud without imposing real-time requirements on the application servers. This is both an architectural advantage (non-real-time systems always being simpler than real-time ones, on many axes) and a user experience benefit.
> 
> 
> 
> It's hard to quantify the benefit and unnecessary to the justification, but nevertheless I am working on this.
> 
> 
> 
> So, I agree with steps 1-3 and they have been completed (modulo restoration of material you have deleted from the wiki).
> 
> 
> 
> I disagree that steps 1-3 have been completed. Specifically, this thread has raised new information. As I said above, I think steps 1-2 are are prerequisites for [4].
> 
> 
> 
> The old wiki text is all there in the history. However, I highly doubt simply reverting the deletion addresses these steps. We need sufficient detail to understand the use cases, implementation requirements, algorithms, and whether delayed shutdown might reappear as a requirement on some platforms.
> 
> 
> 
> ​I added the following here: https://www.w3.org/wiki/HTML/Media_Task_Force/EME_Use_Cases#Failed_Handshake <https://www.w3.org/wiki/HTML/Media_Task_Force/EME_Use_Cases#Failed_Handshake>
> 
> 
> "It is not required to attempt the handshake except by the explicit procedure described above (for example if the web application is closed or crashes)."​
> 
> 
> 
> 
> 
> 4. The group discusses and attempts to reach consensus on a solution for #1.
> 
> 
> 
> ​You are treating this as a brand new feature request wheras I consider it a feature that has been present since day one. It is already supported by two User Agents. Whilst concerns have been expressed previously, the outright opposition is ver​y new. I had expected those earlier concerns to be addressed through the interaction between implementation experience and specification development. Given the implementation status we are ready to complete the specification work. Just because your position changes from concerns (which I am addressing in this very thread as well as before) to opposition does not set back the feature to a square one requirements discussion.
> 
> 
> 
> The definition you appear to be proposing is brand new
> 
> ​.​
> 
>  Earlier this month, you were arguing for a different solution. If some of this information is new to me and the public discussion, I find it hard to believe “It is already supported by two User Agents.”
> 
> 
> 
> ​Well, it is indeed supported and deployed. You can look at the API interactions between our JS on netflix.com <http://netflix.com/> and the browsers in question if you find it hard to believe.
> 
> 
> 
> As noted above, I'm just proposing additional detail about how the feature is implemented under the covers. Whilst previous text talks about persisting the license release message, it is always the case that ​you can implement however you like so long as the API behavior is the same. So, it's just a different way of describing the same thing - actually with a bit more detail. We can go back to the previous way if you like, but I do think this way is better.
> 
> 
> 
> Not to mention, there is no clear definition of what “it” is. The spec process is more than just documenting existing implementations. Features aren’t typically dictated by a single site, either. Perhaps there are others that would like similar functionality, but they would be excluded by the rubber-stamping of such privately-developed definitions.
> 
> 
> 
> ​Well, sites are restricted to using the DRMs supported by the User Agents​. Where these DRMs already support a feature in a common way, I think it makes sense for us to support that in our specification.
> 
> 
> 
> 
> 
> The spec started as a proposal for some simple APIs to perform license exchanges. Through the spec process, including input from WG members and the TAG, the importance of interoperability, fragmentation, implementability, privacy, security, etc. have all become clearer. We've also learned of additional use cases and discussed various user agent implementations. Through this process, the bar for features and functionality has been raised (to be consistent with the general spec process). Having never been concretely defined and posing a number of concerns, it seems reasonable for this feature to be expected to reach this bar.
> 
> 
> 
> ​I'm glad you bring up this evolution in our process which has largely been driven in this group by you. I agree with the direction and I think it would help if we could expand that to explicit agreement of the whole group. We should discuss in more detail what this means for our work, because I think you are making assumptions that are not shared by the group (at least, not yet).
> 
> 
> 
> I'm more than happy to expand the use-case, definition and specification of the feature to meet a higher bar. We should do the same for other features. I think we should just get to work collaborating on that, rather than ripping (certain) things out and starting from scratch. I think that was the conclusion of the F2F.
> 
> 
> 
> 
> 
> We have opposed this feature “since the beginning of the work.” At one point, it was "allowed" due to underspecification. Five months ago, a session type was added because we needed to specify behavior for a different use case ("persistent-license"). A month ago, I accidentally specified the remove() behavior was the same for both session types.
> 
> 
> 
> ​I have seen this problem many times in standardization: you add something to the specification "by mistake" which others think was deliberate and agree with. People read it, even implement based on that. Later you realize your mistake but at this point it is too late. You need to discuss the reversion, why it was a mistake, consider that perhaps even if it was unintentional, it was not actually a mistake for the specification, since it passed group review etc.
> 
> 
> 
> 
> 
> We have always strongly objected to any scenario that involved delaying shutdown. There was a period of time where we were led to believe various things were no longer required, at which time our opposition was lessened. However, that turned out to be false and/or subject to change, and our opposition returned. You'll have to forgive me if I'm cautious about any new claims that delaying shutdown is off the table.
> 
> 
> 
> ​Fair enough. See above.​
> 
> 
> 
> 
> 
> ​...Mark​
> 
> 
> 
> 
> 
> 
> 
> It’s worth noting that without such evolution to the group's approach to the spec and our consistent pushback, delayed shutdown might well be a de facto requirement for all implementations, including desktop browsers. If we are now at a point (different from even earlier this month) where that is no longer being considered, then I would call that a success for the web platform.
> 
> 
> 
> 
> 
> ...Mark
> 
> 
> 
> 
> 
> [1] https://github.com/w3c/encrypted-media/issues/45 <https://github.com/w3c/encrypted-media/issues/45>
> [2] https://github.com/w3c/encrypted-media/issues/45#issuecomment-91743387 <https://github.com/w3c/encrypted-media/issues/45#issuecomment-91743387>
> [3] https://lists.w3.org/Archives/Public/public-html-media/2015Apr/0067.html <https://lists.w3.org/Archives/Public/public-html-media/2015Apr/0067.html>
> 
> 
> ​[4] http://www.w3.org/2015/04/16-html-media-minutes.html#action01] <http://www.w3.org/2015/04/16-html-media-minutes.html#action01]>​
> 
> 
> 
> 
> 
> David
> 
> 
> 
> P.S.
> 
> "Record of license destruction" was only intended for “persistent-license”,
> 
> 
> 
> ​Whatever your intention, what was actually in the specification until your recent changes applied this to both ​“persistent-license” and “persistent-release”.
> 
> 
> 
> There was one inadvertent case where this was the case. It was added a month ago in the commit to ‘Define the behavior for "persistent-license" sessions’ [7]. In your original email, you said it was incorrect for your use case. Fixing this mistake and separating the session types still seems like the appropriate action.
> 
> 
> 
> but it is similarly restrictive for implementations. I’ll change it to something like "persist an invalidation of the license." Other wording suggestions welcome.
> 
> 
> 
> [5] https://github.com/w3c/encrypted-media/commit/71e4352d4a737f3cdba879217dc9daf74020751d <https://github.com/w3c/encrypted-media/commit/71e4352d4a737f3cdba879217dc9daf74020751d>
> [6] https://dvcs.w3.org/hg/html-media/raw-file/eme-v0.1/encrypted-media/encrypted-media.html#key-release <https://dvcs.w3.org/hg/html-media/raw-file/eme-v0.1/encrypted-media/encrypted-media.html#key-release>
> [7] https://github.com/w3c/encrypted-media/commit/76c48b4f0a171af89ed93c21745900ecbb75f7a2 <https://github.com/w3c/encrypted-media/commit/76c48b4f0a171af89ed93c21745900ecbb75f7a2>
> 
> 
> 
> 
> 
> 
> 
> On Tue, Apr 28, 2015 at 8:02 AM, Mark Watson <watsonm@netflix.com <mailto:watsonm@netflix.com>> wrote:
> 
> All,
> 
> 
> 
> Hopefully this will catch you in your newly-freed "EME hour" :-)
> 
> 
> 
> As promised at the F2F I will draft spec updates this week to fill in the missing details of this feature. However, I would like to make one significant change in response to comments at the F2F and previously regarding exactly what is persisted.
> 
> 
> 
> The spec describes persistence of a "record of license destruction". Aside from the fact that we have no concept in our spec of "license", this is not typically how this feature is implemented in DRMs and suggests a need to persist data at close / shutdown / crash or other events that cause license destruction.
> 
> 
> 
> In practice, what is persisted  is a record of available keys at a point in time. This is updated regularly *during streaming*. There is no update on close / shutdown / crash. In a later browsing session, this record is compared with the actually available keys and a discrepancy is taken as evidence that keys were destroyed.
> 
> 
> 
> I propose that we describe this persisted data explicitly as "for each key in the session, the first decrypt time and the latest decrypt time".*
> 
> 
> 
> Are there any comments on that before I implement it ?
> 
> 
> 
> Based on this change in description, I suggest we call this session type "tracked" - or something similar - since really we are 'tracking' the usage of keys. This name will also invite appropriate scrutiny of the persisted data properties.
> 
> 
> 
> ...Mark
> 
> 
> 
> * there may of course be other CDM-specific information persisted and included in the release message. For example if the CDM has a concept of licenses, then license correlation identifiers may be present. Also, how the CDM communicates time in its messages may be CDM-specific.
> 
> 
> 
> 
> 
> 
> 
> 
> 
> 
> 
> 
> 
> 
> 
> 
>