Re: Secure release and persistence

On Wed, Apr 29, 2015 at 5:37 PM, David Dorwin <ddorwin@google.com> wrote:

> The tone of this thread has turned for the worse. I've addressed some of
> Mark’s comments below, but I'll wait for #1 and #2 before replying further.
>
> On Wed, Apr 29, 2015 at 9:57 AM, Mark Watson <watsonm@netflix.com> wrote:
>
>>
>>
>> On Wed, Apr 29, 2015 at 9:29 AM, David Dorwin <ddorwin@google.com> wrote:
>>
>>> While I welcome further clarification on Netflix's feature, I object to
>>> further specifying it in the spec for the reasons described in issue 45
>>> [1], specifically [2].
>>>
>>
>> ​Well, the F2F agreed to give me an action to provide the remaining
>> specification details [4] and I in turn object to the removal of much of
>> the existing specification of this feature, which you did last week under
>> the guise of a "name change".
>>
>
> I believe #1 and #2 are consistent with [4]. It doesn't make sense to
> “provide... technical recommendations” for a problem that isn't fully
> defined.
>
> I’m not sure why you think I removed “much of the existing specification
> of this feature.” Perhaps because there wasn't much actually specified,
> minor deletions appear to be “much” of the specification. I assume you are
> referring to the change to remove() (see below) or [5]. The main changes in
> [5] other than the name were the removal of a) “generate a persistable
> proof of license release”, which was incorrect, and b) specification of
> what the license passed to update() must contain, which depends on further
> definition of the feature. Both were replaced with "*TBD*."
>

​Yes. The feature doesn't need much in the spec and those things
constituted a major part of it.​

​I restored the material in the wiki:
https://www.w3.org/wiki/HTML/Media_Task_Force/EME_Use_Cases#Limited_Concurrent_Streams_via_Key_Release

I also made a few updates ​to align with what was in the spec before the
recent changes.

Honestly, the level of detail for this feature in the Use-Cases wiki is not
different from all the other "supported" features. I'd encourage all task
force members to look at it and tell me if I am going blind somehow. I'll
happily expand it, but perhaps we should do the same for all the features,
not just this one ? (see below on this).

I really don't think it's acceptable for a task force member (editor or
not) to change a use-case from "Supported" to "Not supported" (or vice
versa) when that has *not been agreed by the group* and indeed is an active
topic of discussion. This wiki of use-cases has been stable for a while and
additions / removals should be by agreement.


>
>
>>
>>
>>> There is still no data justifying the need to add this new requirement
>>> to all implementations
>>>
>>
>> ​It's optional.​
>>
>
> I disagree with this justification. As discussed at the f2f, optional
> features are considered an anti-pattern because they either become required
> to remain competitive or are never implemented. In this case, “optional” is
> de facto required because Netflix (may) require it.
>

​I was just pointing out that "​new requirement to all implementations" was
not accurate.

The whole issue of optional features deserves more consideration. This is
not the only one. It is not obvious, and it's feature-dependent, that an
optional feature becomes *de facto* required or nowhere implemented. One
can easily imagine features that enhance user experience where UAs disagree
on the competitive value vs other things they might implement. Pre-fetching
of licenses without a <video> element is a good example. So, that it is
always an "anti-pattern" is your opinion, not something we all agree on.

Since this issue applies to multiple features, can we leave it here for the
moment, until we have addressed the other issues ?


> Regardless, there is no data justifying even the “optional” feature and
> the resulting fragmentation.
>
>>
>>
>>> nor has there been consideration of available alternatives.
>>>
>>
>> ​License renewal is an alternative which has been extensively discussed,
>> has significant dis-advantages and also remains underspecified ​
>>
>
> I said consideration, not discussion. Specifically, there has been no
> experimentation with the already available feature set. Only your
> assertions that they will not work and Netflix will not consider them.
>

​Secure release is part of the ​"already available feature set" as far as I
am concerned and we have done more than experiment. It is deployed at scale
with Safari and IE and it is working well.

Is license renewal in use on desktop browsers by any service providers ? By
whom ? Perhaps we can compare notes ?


>
>>
>>> Both are important prerequisites for adding features to the web
>>> platform. Furthermore, Netflix's stated use case - whether mentioned in the
>>> spec or not - requires a fundamental violation of the web platform
>>> architecture: the CDM/media stack delaying application shutdown.
>>>
>>
>> ​No, this is not required by our use-case and the more detailed
>> specification I will provide this week will make that even clearer, as
>> detailed at the top of this thread.​
>>
>
> Are you saying that with this new solution, Netflix will not require *any*
> user agents to delay shutdown and ensure that messages reach the server?
>

​It's not a "new solution" - it's just more detail on how everyone
implemented what was there before.

Delaying shutdown is not mentioned in the use-case wiki, it's not mentioned
in the specification, we don't require it for our use-case and responsive
to your concerns I'm *trying* to find ways to make this clearer in the
spec. If you can find a way for the specification to *prohibit* it, that
would be fine by me. We don't require any user agents to delay shutdown.
What else can I do to address your concern ?


>
>>
>>>
>>> Based on these important concerns, clear lack of consensus, continuing
>>> confusion about this feature even among the editors [3], and lack of data
>>> or experimentation, it is still too early to include this in the spec and
>>> require it of all implementations.
>>>
>>> Even with Mark’s description below, there is still confusion about this
>>> feature [3]. I propose the following next steps for Mark’s feature request:
>>> 1. Mark clearly defines (all) use case(s) for this feature (perhaps in
>>> the wiki or a doc).
>>> 2. Mark clearly defines the feature he is requesting (perhaps in the
>>> wiki or a doc).
>>>
>>
>> ​The use-case and feature definition have been present, first in the
>> original specification and later in the wiki, since the beginning of the
>> work (i.e. several years). I'm happy to refresh this as I understand you
>> may have removed some of this from the wiki recently.
>>
>
> The use case has been unclear and there has never been a complete feature
> definition. As an example, see the text from “the beginning of the work”:
> [6]. Also, as far as I am aware, this thread is the first time that saving
> decrypt times has been discussed publicly.
>

​The use-case is no less clear than all the others (see below).​

​Yes, I'm proposing an additional level of detail based on how everyone has
implemented this which, since it is common, can be described in the
specification. It's good for implementation experience to inform the spec.
I thought it would help with your concerns, since it makes it clear there
is nothing to be done / persisted / sent etc. at page close.​


>
> Yes, I removed the text from the wiki and added a note that the model has
> not been explicitly defined. The text, much of which I had written, was out
> of date, and I did not want it to add confusion to the ongoing discussion.
>
>>
>>
>>> 3. Mark provides justification for adding this feature, including data
>>> on alternatives considered and experimented with as well as user benefit of
>>> this feature over alternatives.
>>>
>>
>> ​As I have already explained several times, the user benefit of this
>> feature over license renewal is that streams-in-progress do not fail when
>> there is a loss of connectivity to the application server or an application
>> server outage.​ Put another way, this feature enables detection of
>> concurrent stream limit fraud without imposing real-time requirements on
>> the application servers. This is both an architectural advantage
>> (non-real-time systems always being simpler than real-time ones, on many
>> axes) and a user experience benefit.
>>
>> It's hard to quantify the benefit and unnecessary to the justification,
>> but nevertheless I am working on this.
>>
>> So, I agree with steps 1-3 and they have been completed (modulo
>> restoration of material you have deleted from the wiki).
>>
>
> I disagree that steps 1-3 have been completed. Specifically, this thread
> has raised new information. As I said above, I think steps 1-2 are are
> prerequisites for [4].
>
> The old wiki text is all there in the history. However, I highly doubt
> simply reverting the deletion addresses these steps. We need sufficient
> detail to understand the use cases, implementation requirements,
> algorithms, and whether delayed shutdown might reappear as a requirement on
> some platforms.
>

​I added the following here:
https://www.w3.org/wiki/HTML/Media_Task_Force/EME_Use_Cases#Failed_Handshake

"It is not required to attempt the handshake except by the explicit
procedure described above (for example if the web application is closed or
crashes)."​


>
>>
>>> 4. The group discusses and attempts to reach consensus on a solution for
>>> #1.
>>>
>>
>> ​You are treating this as a brand new feature request wheras I consider
>> it a feature that has been present since day one. It is already supported
>> by two User Agents. Whilst concerns have been expressed previously, the
>> outright opposition is ver​y new. I had expected those earlier concerns to
>> be addressed through the interaction between implementation experience and
>> specification development. Given the implementation status we are ready to
>> complete the specification work. Just because your position changes from
>> concerns (which I am addressing in this very thread as well as before) to
>> opposition does not set back the feature to a square one requirements
>> discussion.
>>
>
> The definition you appear to be proposing is brand new
> ​.​
>  Earlier this month, you were arguing for a different solution. If some of
> this information is new to me and the public discussion, I find it hard to
> believe “It is already supported by two User Agents.”
>

​Well, it is indeed supported and deployed. You can look at the API
interactions between our JS on netflix.com and the browsers in question if
you find it hard to believe.

As noted above, I'm just proposing additional detail about how the feature
is implemented under the covers. Whilst previous text talks about
persisting the license release message, it is always the case that ​you can
implement however you like so long as the API behavior is the same. So,
it's just a different way of describing the same thing - actually with a
bit more detail. We can go back to the previous way if you like, but I do
think this way is better.


> Not to mention, there is no clear definition of what “it” is. The spec
> process is more than just documenting existing implementations. Features
> aren’t typically dictated by a single site, either. Perhaps there are
> others that would like similar functionality, but they would be excluded by
> the rubber-stamping of such privately-developed definitions.
>

​Well, sites are restricted to using the DRMs supported by the User
Agents​. Where these DRMs already support a feature in a common way, I
think it makes sense for us to support that in our specification.


>
> The spec started as a proposal for some simple APIs to perform license
> exchanges. Through the spec process, including input from WG members and
> the TAG, the importance of interoperability, fragmentation,
> implementability, privacy, security, etc. have all become clearer. We've
> also learned of additional use cases and discussed various user agent
> implementations. Through this process, the bar for features and
> functionality has been raised (to be consistent with the general spec
> process). Having never been concretely defined and posing a number of
> concerns, it seems reasonable for this feature to be expected to reach this
> bar.
>

​I'm glad you bring up this evolution in our process which has largely been
driven in this group by you. I agree with the direction and I think it
would help if we could expand that to explicit agreement of the whole
group. We should discuss in more detail what this means for our work,
because I think you are making assumptions that are not shared by the group
(at least, not yet).

I'm more than happy to expand the use-case, definition and specification of
the feature to meet a higher bar. We should do the same for other features.
I think we should just get to work collaborating on that, rather than
ripping (certain) things out and starting from scratch. I think that was
the conclusion of the F2F.


>
> We have opposed this feature “since the beginning of the work.” At one
> point, it was "allowed" due to underspecification. Five months ago, a
> session type was added because we needed to specify behavior for a
> different use case ("persistent-license"). A month ago, I accidentally
> specified the remove() behavior was the same for both session types.
>

​I have seen this problem many times in standardization: you add something
to the specification "by mistake" which others think was deliberate and
agree with. People read it, even implement based on that. Later you realize
your mistake but at this point it is too late. You need to discuss the
reversion, why it was a mistake, consider that perhaps even if it was
unintentional, it was not *actually* a mistake for the specification, since
it passed group review etc.


>
> We have always strongly objected to any scenario that involved delaying
> shutdown. There was a period of time where we were led to believe various
> things were no longer required, at which time our opposition was lessened.
> However, that turned out to be false and/or subject to change, and our
> opposition returned. You'll have to forgive me if I'm cautious about any
> new claims that delaying shutdown is off the table.
>

​Fair enough. See above.​


​...Mark​


>
> It’s worth noting that without such evolution to the group's approach to
> the spec and our consistent pushback, delayed shutdown might well be a de
> facto requirement for all implementations, including desktop browsers. If
> we are now at a point (different from even earlier this month) where that
> is no longer being considered, then I would call that a success for the web
> platform.
>
>
>> ...Mark
>>
>>
>>> [1] https://github.com/w3c/encrypted-media/issues/45
>>> [2]
>>> https://github.com/w3c/encrypted-media/issues/45#issuecomment-91743387
>>> [3]
>>> https://lists.w3.org/Archives/Public/public-html-media/2015Apr/0067.html
>>>
>>
>> ​[4] http://www.w3.org/2015/04/16-html-media-minutes.html#action01]​
>>
>>
>>>
>>> David
>>>
>>> P.S.
>>> "Record of license destruction" was only intended for
>>> “persistent-license”,
>>>
>>
>> ​Whatever your intention, what was actually in the specification until
>> your recent changes applied this to both ​“persistent-license” and
>> “persistent-release”.
>>
>
> There was one inadvertent case where this was the case. It was added a
> month ago in the commit to ‘Define the behavior for "persistent-license"
> sessions’ [7]. In your original email, you said it was incorrect for your
> use case. Fixing this mistake and separating the session types still seems
> like the appropriate action.
>
>>
>>
>>> but it is similarly restrictive for implementations. I’ll change it to
>>> something like "persist an invalidation of the license." Other wording
>>> suggestions welcome.
>>>
>>
>> [5]
> https://github.com/w3c/encrypted-media/commit/71e4352d4a737f3cdba879217dc9daf74020751d
> [6]
> https://dvcs.w3.org/hg/html-media/raw-file/eme-v0.1/encrypted-media/encrypted-media.html#key-release
> [7]
> https://github.com/w3c/encrypted-media/commit/76c48b4f0a171af89ed93c21745900ecbb75f7a2
>
>
>>
>>
>>
>>>
>>> On Tue, Apr 28, 2015 at 8:02 AM, Mark Watson <watsonm@netflix.com>
>>> wrote:
>>>
>>>> All,
>>>>
>>>> Hopefully this will catch you in your newly-freed "EME hour" :-)
>>>>
>>>> As promised at the F2F I will draft spec updates this week to fill in
>>>> the missing details of this feature. However, I would like to make one
>>>> significant change in response to comments at the F2F and previously
>>>> regarding exactly what is persisted.
>>>>
>>>> The spec describes persistence of a "record of license destruction".
>>>> Aside from the fact that we have no concept in our spec of "license", this
>>>> is not typically how this feature is implemented in DRMs and suggests a
>>>> need to persist data at close / shutdown / crash or other events that cause
>>>> license destruction.
>>>>
>>>> In practice, what is persisted  is a record of available keys at a
>>>> point in time. This is updated regularly *during streaming*. There is no
>>>> update on close / shutdown / crash. In a later browsing session, this
>>>> record is compared with the actually available keys and a discrepancy is
>>>> taken as evidence that keys were destroyed.
>>>>
>>>> I propose that we describe this persisted data explicitly as "for each
>>>> key in the session, the *first decrypt time* and the *latest decrypt
>>>> time*".*
>>>>
>>>> Are there any comments on that before I implement it ?
>>>>
>>>> Based on this change in description, I suggest we call this session
>>>> type "*tracked*" - or something similar - since really we are
>>>> 'tracking' the usage of keys. This name will also invite appropriate
>>>> scrutiny of the persisted data properties.
>>>>
>>>> ...Mark
>>>>
>>>> * there may of course be other CDM-specific information persisted and
>>>> included in the release message. For example if the CDM has a concept of
>>>> licenses, then license correlation identifiers may be present. Also, how
>>>> the CDM communicates time in its messages may be CDM-specific.
>>>>
>>>>
>>>>
>>>>
>>>
>>
>