Re: Secure release and persistence

David,

Regarding "delayed shutdown", whilst I think we are on the same page, I
think it would be good if you could define more precisely the behaviors you
would like to avoid and the architectural principle you would like to see
maintained. I don't want us to miss any differences in understanding now -
either between us or across browsers - that might cause trouble further
down the road.

Thanks,

...Mark



On Thu, Apr 30, 2015 at 3:29 PM, Mark Watson <watsonm@netflix.com> wrote:

> Here is a Pull Request in fulfilment of my Action from the F2F:
> https://github.com/w3c/encrypted-media/pull/54
>
> You can see the updated specification at
> http://mwatson2.github.io/encrypted-media/
>
> I will provide a description of the changes in Issue#45 and update the
> use-case wiki with further information as requested.
>
> ...Mark
>
> On Thu, Apr 30, 2015 at 2:59 PM, Jerry Smith (WINDOWS) <
> jdsmith@microsoft.com> wrote:
>
>>  I think the tone of this email has been relatively civil given some of
>> the strong opposing opinions involved.  One thing seems to be clearly
>> stated now that maybe wasn’t before:  Mark has confirmed that Netflix has
>> no plans to insist on delayed browser close/automatic messaging, and they
>> are open to spec language that states this.
>>
>>
>>
>> There still appears to be significant concern about the feature
>> maintaining its optional status.  I’d be interested in others comments on
>> this.  My own take is that the non-real time aspect of the feature make it
>> poorly suited to binding features to its presence.   I’ve previously said
>> that having it supported on a subset of UAs can still add value, with no
>> feature binding involved.  Broad UA support is preferable, but doesn’t seem
>> a requirement for the feature to be justified.
>>
>>
>>
>> Jerry
>>
>>
>>
>> *From:* Mark Watson [mailto:watsonm@netflix.com]
>> *Sent:* Thursday, April 30, 2015 11:12 AM
>> *To:* David Dorwin
>> *Cc:* public-html-media@w3.org
>> *Subject:* Re: Secure release and persistence
>>
>>
>>
>>
>>
>>
>>
>> On Wed, Apr 29, 2015 at 5:37 PM, David Dorwin <ddorwin@google.com> wrote:
>>
>>  The tone of this thread has turned for the worse. I've addressed some
>> of Mark’s comments below, but I'll wait for #1 and #2 before replying
>> further.
>>
>>
>>
>> On Wed, Apr 29, 2015 at 9:57 AM, Mark Watson <watsonm@netflix.com> wrote:
>>
>>
>>
>>
>>
>> On Wed, Apr 29, 2015 at 9:29 AM, David Dorwin <ddorwin@google.com> wrote:
>>
>>  While I welcome further clarification on Netflix's feature, I object to
>> further specifying it in the spec for the reasons described in issue 45
>> [1], specifically [2].
>>
>>
>>
>> ​Well, the F2F agreed to give me an action to provide the remaining
>> specification details [4] and I in turn object to the removal of much of
>> the existing specification of this feature, which you did last week under
>> the guise of a "name change".
>>
>>
>>
>> I believe #1 and #2 are consistent with [4]. It doesn't make sense to
>> “provide... technical recommendations” for a problem that isn't fully
>> defined.
>>
>>
>>
>> I’m not sure why you think I removed “much of the existing specification
>> of this feature.” Perhaps because there wasn't much actually specified,
>> minor deletions appear to be “much” of the specification. I assume you are
>> referring to the change to remove() (see below) or [5]. The main changes in
>> [5] other than the name were the removal of a) “generate a persistable
>> proof of license release”, which was incorrect, and b) specification of
>> what the license passed to update() must contain, which depends on further
>> definition of the feature. Both were replaced with "*TBD*."
>>
>>
>>
>> ​Yes. The feature doesn't need much in the spec and those things
>> constituted a major part of it.​
>>
>>
>>
>> ​I restored the material in the wiki:
>> https://www.w3.org/wiki/HTML/Media_Task_Force/EME_Use_Cases#Limited_Concurrent_Streams_via_Key_Release
>>
>>
>>
>> I also made a few updates ​to align with what was in the spec before the
>> recent changes.
>>
>>
>>
>> Honestly, the level of detail for this feature in the Use-Cases wiki is
>> not different from all the other "supported" features. I'd encourage all
>> task force members to look at it and tell me if I am going blind somehow.
>> I'll happily expand it, but perhaps we should do the same for all the
>> features, not just this one ? (see below on this).
>>
>>
>>
>> I really don't think it's acceptable for a task force member (editor or
>> not) to change a use-case from "Supported" to "Not supported" (or vice
>> versa) when that has *not been agreed by the group* and indeed is an
>> active topic of discussion. This wiki of use-cases has been stable for a
>> while and additions / removals should be by agreement.
>>
>>
>>
>>
>>
>>
>>
>>  There is still no data justifying the need to add this new requirement
>> to all implementations
>>
>>
>>
>> ​It's optional.​
>>
>>
>>
>> I disagree with this justification. As discussed at the f2f, optional
>> features are considered an anti-pattern because they either become required
>> to remain competitive or are never implemented. In this case, “optional” is
>> de facto required because Netflix (may) require it.
>>
>>
>>
>> ​I was just pointing out that "​new requirement to all implementations"
>> was not accurate.
>>
>>
>>
>> The whole issue of optional features deserves more consideration. This is
>> not the only one. It is not obvious, and it's feature-dependent, that an
>> optional feature becomes *de facto* required or nowhere implemented. One
>> can easily imagine features that enhance user experience where UAs disagree
>> on the competitive value vs other things they might implement. Pre-fetching
>> of licenses without a <video> element is a good example. So, that it is
>> always an "anti-pattern" is your opinion, not something we all agree on.
>>
>>
>>
>> Since this issue applies to multiple features, can we leave it here for
>> the moment, until we have addressed the other issues ?
>>
>>
>>
>>
>>
>> Regardless, there is no data justifying even the “optional” feature and
>> the resulting fragmentation.
>>
>>
>>
>>  nor has there been consideration of available alternatives.
>>
>>
>>
>> ​License renewal is an alternative which has been extensively discussed,
>> has significant dis-advantages and also remains underspecified ​
>>
>>
>>
>> I said consideration, not discussion. Specifically, there has been no
>> experimentation with the already available feature set. Only your
>> assertions that they will not work and Netflix will not consider them.
>>
>>
>>
>> ​Secure release is part of the ​"already available feature set" as far as
>> I am concerned and we have done more than experiment. It is deployed at
>> scale with Safari and IE and it is working well.
>>
>>
>>
>> Is license renewal in use on desktop browsers by any service providers ?
>> By whom ? Perhaps we can compare notes ?
>>
>>
>>
>>
>>
>>  Both are important prerequisites for adding features to the web
>> platform. Furthermore, Netflix's stated use case - whether mentioned in the
>> spec or not - requires a fundamental violation of the web platform
>> architecture: the CDM/media stack delaying application shutdown.
>>
>>
>>
>> ​No, this is not required by our use-case and the more detailed
>> specification I will provide this week will make that even clearer, as
>> detailed at the top of this thread.​
>>
>>
>>
>> Are you saying that with this new solution, Netflix will not require
>> *any* user agents to delay shutdown and ensure that messages reach the
>> server?
>>
>>
>>
>> ​It's not a "new solution" - it's just more detail on how everyone
>> implemented what was there before.
>>
>>
>>
>> Delaying shutdown is not mentioned in the use-case wiki, it's not
>> mentioned in the specification, we don't require it for our use-case and
>> responsive to your concerns I'm *trying* to find ways to make this clearer
>> in the spec. If you can find a way for the specification to *prohibit* it,
>> that would be fine by me. We don't require any user agents to delay
>> shutdown. What else can I do to address your concern ?
>>
>>
>>
>>
>>
>>
>>
>> Based on these important concerns, clear lack of consensus, continuing
>> confusion about this feature even among the editors [3], and lack of data
>> or experimentation, it is still too early to include this in the spec and
>> require it of all implementations.
>>
>>
>>
>> Even with Mark’s description below, there is still confusion about this
>> feature [3]. I propose the following next steps for Mark’s feature request:
>>
>> 1. Mark clearly defines (all) use case(s) for this feature (perhaps in
>> the wiki or a doc).
>>
>> 2. Mark clearly defines the feature he is requesting (perhaps in the wiki
>> or a doc).
>>
>>
>>
>> ​The use-case and feature definition have been present, first in the
>> original specification and later in the wiki, since the beginning of the
>> work (i.e. several years). I'm happy to refresh this as I understand you
>> may have removed some of this from the wiki recently.
>>
>>
>>
>> The use case has been unclear and there has never been a complete feature
>> definition. As an example, see the text from “the beginning of the work”:
>> [6]. Also, as far as I am aware, this thread is the first time that saving
>> decrypt times has been discussed publicly.
>>
>>
>>
>> ​The use-case is no less clear than all the others (see below).​
>>
>>
>>
>> ​Yes, I'm proposing an additional level of detail based on how everyone
>> has implemented this which, since it is common, can be described in the
>> specification. It's good for implementation experience to inform the spec.
>> I thought it would help with your concerns, since it makes it clear there
>> is nothing to be done / persisted / sent etc. at page close.​
>>
>>
>>
>>
>>
>> Yes, I removed the text from the wiki and added a note that the model has
>> not been explicitly defined. The text, much of which I had written, was out
>> of date, and I did not want it to add confusion to the ongoing discussion.
>>
>>
>>
>>  3. Mark provides justification for adding this feature, including data
>> on alternatives considered and experimented with as well as user benefit of
>> this feature over alternatives.
>>
>>
>>
>> ​As I have already explained several times, the user benefit of this
>> feature over license renewal is that streams-in-progress do not fail when
>> there is a loss of connectivity to the application server or an application
>> server outage.​ Put another way, this feature enables detection of
>> concurrent stream limit fraud without imposing real-time requirements on
>> the application servers. This is both an architectural advantage
>> (non-real-time systems always being simpler than real-time ones, on many
>> axes) and a user experience benefit.
>>
>>
>>
>> It's hard to quantify the benefit and unnecessary to the justification,
>> but nevertheless I am working on this.
>>
>>
>>
>> So, I agree with steps 1-3 and they have been completed (modulo
>> restoration of material you have deleted from the wiki).
>>
>>
>>
>> I disagree that steps 1-3 have been completed. Specifically, this thread
>> has raised new information. As I said above, I think steps 1-2 are are
>> prerequisites for [4].
>>
>>
>>
>> The old wiki text is all there in the history. However, I highly doubt
>> simply reverting the deletion addresses these steps. We need sufficient
>> detail to understand the use cases, implementation requirements,
>> algorithms, and whether delayed shutdown might reappear as a requirement on
>> some platforms.
>>
>>
>>
>> ​I added the following here:
>> https://www.w3.org/wiki/HTML/Media_Task_Force/EME_Use_Cases#Failed_Handshake
>>
>>
>>
>> "It is not required to attempt the handshake except by the explicit
>> procedure described above (for example if the web application is closed or
>> crashes)."​
>>
>>
>>
>>
>>
>>  4. The group discusses and attempts to reach consensus on a solution
>> for #1.
>>
>>
>>
>> ​You are treating this as a brand new feature request wheras I consider
>> it a feature that has been present since day one. It is already supported
>> by two User Agents. Whilst concerns have been expressed previously, the
>> outright opposition is ver​y new. I had expected those earlier concerns to
>> be addressed through the interaction between implementation experience and
>> specification development. Given the implementation status we are ready to
>> complete the specification work. Just because your position changes from
>> concerns (which I am addressing in this very thread as well as before) to
>> opposition does not set back the feature to a square one requirements
>> discussion.
>>
>>
>>
>> The definition you appear to be proposing is brand new
>>
>> ​.​
>>
>>  Earlier this month, you were arguing for a different solution. If some
>> of this information is new to me and the public discussion, I find it hard
>> to believe “It is already supported by two User Agents.”
>>
>>
>>
>> ​Well, it is indeed supported and deployed. You can look at the API
>> interactions between our JS on netflix.com and the browsers in question
>> if you find it hard to believe.
>>
>>
>>
>> As noted above, I'm just proposing additional detail about how the
>> feature is implemented under the covers. Whilst previous text talks about
>> persisting the license release message, it is always the case that ​you can
>> implement however you like so long as the API behavior is the same. So,
>> it's just a different way of describing the same thing - actually with a
>> bit more detail. We can go back to the previous way if you like, but I do
>> think this way is better.
>>
>>
>>
>>   Not to mention, there is no clear definition of what “it” is. The spec
>> process is more than just documenting existing implementations. Features
>> aren’t typically dictated by a single site, either. Perhaps there are
>> others that would like similar functionality, but they would be excluded by
>> the rubber-stamping of such privately-developed definitions.
>>
>>
>>
>> ​Well, sites are restricted to using the DRMs supported by the User
>> Agents​. Where these DRMs already support a feature in a common way, I
>> think it makes sense for us to support that in our specification.
>>
>>
>>
>>
>>
>> The spec started as a proposal for some simple APIs to perform license
>> exchanges. Through the spec process, including input from WG members and
>> the TAG, the importance of interoperability, fragmentation,
>> implementability, privacy, security, etc. have all become clearer. We've
>> also learned of additional use cases and discussed various user agent
>> implementations. Through this process, the bar for features and
>> functionality has been raised (to be consistent with the general spec
>> process). Having never been concretely defined and posing a number of
>> concerns, it seems reasonable for this feature to be expected to reach this
>> bar.
>>
>>
>>
>> ​I'm glad you bring up this evolution in our process which has largely
>> been driven in this group by you. I agree with the direction and I think it
>> would help if we could expand that to explicit agreement of the whole
>> group. We should discuss in more detail what this means for our work,
>> because I think you are making assumptions that are not shared by the group
>> (at least, not yet).
>>
>>
>>
>> I'm more than happy to expand the use-case, definition and specification
>> of the feature to meet a higher bar. We should do the same for other
>> features. I think we should just get to work collaborating on that, rather
>> than ripping (certain) things out and starting from scratch. I think that
>> was the conclusion of the F2F.
>>
>>
>>
>>
>>
>> We have opposed this feature “since the beginning of the work.” At one
>> point, it was "allowed" due to underspecification. Five months ago, a
>> session type was added because we needed to specify behavior for a
>> different use case ("persistent-license"). A month ago, I accidentally
>> specified the remove() behavior was the same for both session types.
>>
>>
>>
>> ​I have seen this problem many times in standardization: you add
>> something to the specification "by mistake" which others think was
>> deliberate and agree with. People read it, even implement based on that.
>> Later you realize your mistake but at this point it is too late. You need
>> to discuss the reversion, why it was a mistake, consider that perhaps even
>> if it was unintentional, it was not *actually* a mistake for the
>> specification, since it passed group review etc.
>>
>>
>>
>>
>>
>> We have always strongly objected to any scenario that involved delaying
>> shutdown. There was a period of time where we were led to believe various
>> things were no longer required, at which time our opposition was lessened.
>> However, that turned out to be false and/or subject to change, and our
>> opposition returned. You'll have to forgive me if I'm cautious about any
>> new claims that delaying shutdown is off the table.
>>
>>
>>
>> ​Fair enough. See above.​
>>
>>
>>
>>
>>
>> ​...Mark​
>>
>>
>>
>>
>>
>>
>>
>> It’s worth noting that without such evolution to the group's approach to
>> the spec and our consistent pushback, delayed shutdown might well be a de
>> facto requirement for all implementations, including desktop browsers. If
>> we are now at a point (different from even earlier this month) where that
>> is no longer being considered, then I would call that a success for the web
>> platform.
>>
>>
>>
>>
>>
>> ...Mark
>>
>>
>>
>>
>>
>> [1] https://github.com/w3c/encrypted-media/issues/45
>>
>> [2]
>> https://github.com/w3c/encrypted-media/issues/45#issuecomment-91743387
>>
>> [3]
>> https://lists.w3.org/Archives/Public/public-html-media/2015Apr/0067.html
>>
>>
>>
>> ​[4] http://www.w3.org/2015/04/16-html-media-minutes.html#action01]​
>>
>>
>>
>>
>>
>> David
>>
>>
>>
>> P.S.
>>
>> "Record of license destruction" was only intended for
>> “persistent-license”,
>>
>>
>>
>> ​Whatever your intention, what was actually in the specification until
>> your recent changes applied this to both ​“persistent-license” and
>> “persistent-release”.
>>
>>
>>
>> There was one inadvertent case where this was the case. It was added a
>> month ago in the commit to ‘Define the behavior for "persistent-license"
>> sessions’ [7]. In your original email, you said it was incorrect for your
>> use case. Fixing this mistake and separating the session types still seems
>> like the appropriate action.
>>
>>
>>
>>  but it is similarly restrictive for implementations. I’ll change it to
>> something like "persist an invalidation of the license." Other wording
>> suggestions welcome.
>>
>>
>>
>>  [5]
>> https://github.com/w3c/encrypted-media/commit/71e4352d4a737f3cdba879217dc9daf74020751d
>>
>> [6]
>> https://dvcs.w3.org/hg/html-media/raw-file/eme-v0.1/encrypted-media/encrypted-media.html#key-release
>>
>> [7]
>> https://github.com/w3c/encrypted-media/commit/76c48b4f0a171af89ed93c21745900ecbb75f7a2
>>
>>
>>
>>
>>
>>
>>
>>
>> On Tue, Apr 28, 2015 at 8:02 AM, Mark Watson <watsonm@netflix.com> wrote:
>>
>>  All,
>>
>>
>>
>> Hopefully this will catch you in your newly-freed "EME hour" :-)
>>
>>
>>
>> As promised at the F2F I will draft spec updates this week to fill in the
>> missing details of this feature. However, I would like to make one
>> significant change in response to comments at the F2F and previously
>> regarding exactly what is persisted.
>>
>>
>>
>> The spec describes persistence of a "record of license destruction".
>> Aside from the fact that we have no concept in our spec of "license", this
>> is not typically how this feature is implemented in DRMs and suggests a
>> need to persist data at close / shutdown / crash or other events that cause
>> license destruction.
>>
>>
>>
>> In practice, what is persisted  is a record of available keys at a point
>> in time. This is updated regularly *during streaming*. There is no update
>> on close / shutdown / crash. In a later browsing session, this record is
>> compared with the actually available keys and a discrepancy is taken as
>> evidence that keys were destroyed.
>>
>>
>>
>> I propose that we describe this persisted data explicitly as "for each
>> key in the session, the *first decrypt time* and the *latest decrypt
>> time*".*
>>
>>
>>
>> Are there any comments on that before I implement it ?
>>
>>
>>
>> Based on this change in description, I suggest we call this session type "
>> *tracked*" - or something similar - since really we are 'tracking' the
>> usage of keys. This name will also invite appropriate scrutiny of the
>> persisted data properties.
>>
>>
>>
>> ...Mark
>>
>>
>>
>> * there may of course be other CDM-specific information persisted and
>> included in the release message. For example if the CDM has a concept of
>> licenses, then license correlation identifiers may be present. Also, how
>> the CDM communicates time in its messages may be CDM-specific.
>>
>>
>>
>>
>>
>>
>>
>>
>>
>>
>>
>>
>>
>>
>>
>
>