Re: Resolving issues around avoiding key requests

To avoid further confusion and parallel threads, I will create a pull
request for Issue 52 and reference this thread. I also proposed this
<https://github.com/w3c/encrypted-media/issues/52#issuecomment-137271913>
in Issue 52.

On Tue, Sep 1, 2015 at 1:53 PM, Joe Steele <steele@adobe.com> wrote:

> I responded to both issue 52 (
> https://github.com/w3c/encrypted-media/issues/52) and issue 41 (
> https://github.com/w3c/encrypted-media/issues/41).  I will respond here
> as well so you have the full context.
>
>
>    1. The only thing a user agent does with initData when it encounters
>    it is queue an "encrypted" event with the initData. It may not extract,
>    derive, or decrypt keys, and it may not provide the initData to the CDM.
>
> I) In the same section, add the following after "Initialization data found
> with the media data is provided to the application in the initData
> attribute of the encrypted event."
>
>> When encountered, it MUST NOT be provided to the CDM and its contents
>> MUST NOT be used by the implementation.
>
>
> These two sections imply that there is confusion on how the CDM is
> supposed to get the initData. I don’t think there is, but I would be fine
> with stating explicitly that the User Agent must not provide it _directly_
> to the CDM. I also do not think there is confusion about what the User
> Agent should do with the initData, but I would be fine with explicitly
> stating that the User Agent is restricted in what it can do with the
> initData (e.g. only “sanitize” and provide to the application via
> “encrypted”).
>
> This text implies that the CDM should not be getting the initData (which I
> do not believe is your intention). I would change it to be more explicit
>  e.g. “When encountered, it MUST NOT be provided _directly_ to the CDM _by
> the User Agent_ and its content MUST NOT be used _directly by the User
> Agent_.”.
>

You are correct. Thank you for the suggestion. I was thinking about better
ways to integrate this in that section. I will create a pull request that I
think clearly captures the intent.

>
>    1. The generateRequest() algorithm always generates a message (unless
>    there is an error)
>
> II) Add the following to the first paragraph of the generateRequest
> algorithm.
>
>> A message will always be generated if the algorithm succeeds and the
>> promise is resolved.
>
>
> This text is directly in conflict with what I have proposed in issue 41 so
> of course I disagree. I believe the generateKeyRequest() algorithm should
> change to reflect use cases where a message does not need to be sent
> because the CDM already has a key that can satisfy the request that would
> be generated. Mark also suggested the name is misleading and should instead
> be something like init() and I agree with that suggestion.
>

Those are both significant changes to the spec - and tracked in a separate
bug. The purpose of Issue 52 is to correct and clarify the existing text.
Doing so doesn't prevent future changes.


>
> Joe
>
> On Aug 17, 2015, at 4:42 PM, David Dorwin <ddorwin@google.com> wrote:
>
> Thank you Mark and Joe for the insight and clarification. I'm glad we
> broke Issue 52 out, as it helps clarify the specific problem (at least for
> me). I agree with Mark that his (A) and (B) are equivalent from an
> application visibility perspective. I also agree with him that the proposed
> text breaks key ID verification. My proposal was just a starting point
> trying to express the items of concern; alternative text proposals are
> welcome, as always.
>
> The intent of both the existing and proposed text is to reinforce what is
> supported by the algorithms. They appear to be serving their purpose since
> there appears to be some confusion or lack of awareness of what the
> algorithms say. However, this discussion has made it clear that such simple
> texts will not work.
>
> I see two related issues in the algorithms:
>
>    1. The only thing a user agent does with initData when it encounters
>    it is queue an "encrypted" event with the initData. It may not extract,
>    derive, or decrypt keys, and it may not provide the initData to the CDM.
>
>
>    1. The generateRequest() algorithm always generates a message (unless
>    there is an error)
>
> As a result, the Issue 41 use case and Jerry's August 10 renewal use case
> are not currently supported. Including "key(s)" did not change this, and
> resolving Issue 52 will not change this. These use cases were never
> supported.
>
>
>
> Given the confusion, I think it is useful to continue to have text that
> reinforces these limitations until the algorithms are updated to support
> such features. I have made another proposal for discussion below, but feel
> free to provide suggestions or other proposals.
>
>
> I) In the same section, add the following after "Initialization data found
> with the media data is provided to the application in the initData
> attribute of the encrypted event."
>
>> When encountered, it MUST NOT be provided to the CDM and its contents
>> MUST NOT be used by the implementation
>
>
>
> II) Add the following to the first paragraph of the generateRequest
> algorithm.
>
>> A message will always be generated if the algorithm succeeds and the
>> promise is resolved.
>
>
>
> David
>
> On Fri, Aug 14, 2015 at 4:49 PM, Joe Steele <steele@adobe.com> wrote:
>
>>
>> It is unclear whether such real world use cases exist.
>>
>>
>> Let me be perfectly clear. This is a core component of the key exchange
>> mechanism used by Adobe Access. We have many real world customers using
>> this mechanism today and many of those would like to move to HTML5
>> eventually.
>>
>> However, in order to make the text more technically sound, I propose we
>> remove "key(s)” from the current sentence and add:
>>
>> Implementations MUST NOT use Initialization Data for anything other than
>>> generating a license request. Specifically, Initialization Data or portions
>>> of it MUST NOT be persisted, used to configure the CDM or session,
>>
>>
>> Implementations should only be persisting Initialization Data  for
>> “persistent” sessions. There is no security or privacy gained by preventing
>> the implementation from caching Initialization Data in that scenario.
>>
>>
>> or used directly by the implementation to extract or derive key(s).
>>
>>
>> I disagree with not allowing keys to be extracted or derived from the
>> Init Data. There is no security or private benefit gained by preventing
>> this. I believe the algorithms should be changed to compensate for the
>> minimal difference in application behavior this could result in.
>>
>> We can append "unless explicitly stated in the algorithms" as appropriate
>> when adding support for use cases (i.e. Issues 41 and 53) that need more
>> flexibility.
>>
>>
>> I don’t think this is a useful addition, since almost every algorithm
>> that processes initData would be an exception.
>>
>> I appreciate all the discussion. It sounds like we are coming closer to
>> consensus.
>>
>> Joe
>>
>> On Aug 14, 2015, at 12:08 PM, David Dorwin <ddorwin@google.com> wrote:
>>
>> For security and interoperability reasons, we have a list of data
>> prohibited from Initialization Data. As there is no support in the
>> algorithms for key extraction or its usage in the client, I added "key(s)"
>> to this list. Preventing a key from being in the opaque blob passed to a
>> server or otherwise used in the generation of a license request - and used
>> for nothing else - is an unintentional consequence. (I had not thought of
>> this case, which is why I thought fixing Issue 52 was dependent on
>> supporting one of the other use cases.)
>>
>> It is unclear whether such real world use cases exist. However, in order
>> to make the text more technically sound, I propose we remove "key(s)" from
>> the current sentence and add:
>>
>>> Implementations MUST NOT use Initialization Data for anything other than
>>> generating a license request. Specifically, Initialization Data or portions
>>> of it MUST NOT be persisted, used to configure the CDM or session, or used
>>> directly by the implementation to extract or derive key(s).
>>
>>
>> We can append "unless explicitly stated in the algorithms" as appropriate
>> when adding support for use cases (i.e. Issues 41 and 53) that need more
>> flexibility.
>>
>> I believe this resolves Issue 52 and concerns about the technical
>> correctness of the current spec text without contradicting (by omission)
>> the algorithms.
>>
>> One comment inline.
>>
>> On Thu, Aug 13, 2015 at 8:08 AM, Bob Lund <B.Lund@cablelabs.com> wrote:
>>
>>> All,
>>>
>>> I agree with Mark W, Joe and Jerry on issue 52, especially the point
>>> already raised about opaque InitData always assumed in the development of
>>> the EME spec and that that is consistent with CENC. Absent a compelling,
>>> specific scenario that breaks interoperability we should remove the
>>> constraint.
>>>
>>
>> The scenario where keys in the initData affects interoperability is when
>> it affects application-visible behavior, especially when such behavior is
>> not defined by the algorithms. Specifically, if keys were to be extracted
>> by the CDM and used for anything other than generating a license request,
>> which is currently the only defined/supported use for |initData|.
>>
>>>
>>> Bob
>>>
>>> From: Mark Watson <watsonm@netflix.com>
>>> Date: Wednesday, August 12, 2015 at 5:19 PM
>>> To: "Jerry Smith (IEP)" <jdsmith@microsoft.com>
>>> Cc: David Dorwin <ddorwin@google.com>, Joe Steele <steele@adobe.com>,
>>> "<public-html-media@w3. org>" <public-html-media@w3.org>
>>> Subject: Re: Resolving issues around avoiding key requests
>>> Resent-From: "<public-html-media@w3. org>" <public-html-media@w3.org>
>>> Resent-Date: Wednesday, August 12, 2015 at 5:20 PM
>>>
>>> All,
>>>
>>> I agree with Joe and Jerry.
>>>
>>> Regarding *Issue 52*: it has always been assumed that the initData is
>>> opaque, that the CDM uses information in the initData to construct a key
>>> request and that the server uses that information to construct a response
>>> containing the content key. Any mechanism that adheres to that model should
>>> be allowed. Clearly, we all expect the opaque portion of the initData to
>>> contain information that identifies the content key. Whether that
>>> information is in the form of a key id or the key itself seems irrelevant
>>> to the specification and to interoperability. Is it allowed for the
>>> initData to contain a partial key and then an id for the remainder ? If it
>>> contains one bit of the key and an id for the remainder is it ok ? If it
>>> contains 127 bits of the key and an identifier for the last bit of the key
>>> is that ok ? In neither case does the initData contain a "key". What huge
>>> problem does that one additional bit of the key cause ?
>>>
>>> I also agree with Joes description of the way this change was introduced
>>> without discussion: I think the onus should be on David to describe a
>>> problematic scenario that this prohibition on keys in initData precludes
>>> before we introduce that prohibition. In the meantime the prohibition
>>> should be removed.
>>>
>>> Regarding *Issue 41*: our model for key exchange messages is that the
>>> number of exchanges needed, and their timing, is up to the CDM. CDMs may
>>> need more than one request / response exchange in order to establish the
>>> keys. They may initiate additional exchanges at any time (e.g. for license
>>> renewal). It seems consistent and unproblematic from an interop pov if the
>>> number of exchanges is zero. It has always been possible that the CDM has
>>> other information (from the initData, from other message exchanges) which
>>> mean there is no exchange necessary. Perhaps the beginning of the content -
>>> unbeknownst to the application - is not encrypted and the CDM learns this
>>> from the initData. It seems arbitrary to require the number of message
>>> exchanges at session start to be non-zero.
>>>
>>> Regarding *Issue 53*: it's not clear to me that changes to the
>>> specification are required to enable this, though I agree it warrants
>>> discussion. Certainly I don't see any reason why this issue should block
>>> either 52 or 41, which stand on their own. There is nothing in the
>>> specification preventing key exchanges from delivering CDM-specific
>>> information to the CDM that has a scope greater than the session in which
>>> it is delivered. Such information would not be deleted when the session is
>>> closed and could be used by other sessions. Again, the model is that the
>>> CDM executes as many message exchanges as it needs (zero or more) whenever
>>> it needs to and signals the status with respect to content keys to the
>>> application via KeyStatus.
>>>
>>> ...Mark
>>>
>>>
>>>
>>>
>>> On Mon, Aug 10, 2015 at 10:37 AM, Jerry Smith (IEP) <
>>> jdsmith@microsoft.com> wrote:
>>>
>>>> Keys in initData is specifically allowed by the ISO specification on
>>>> Common Encryption.  I’ve included a reference on this in a comment on Issue
>>>> #52 below.  I do not believe we should make the EME spec more restrictive
>>>> than the controlling ISO CENC documentation.
>>>>
>>>>
>>>>
>>>> Keys distributed in initData support license renewal use cases with
>>>> seamless and efficient key distribution.  This method is commonly used by
>>>> streaming media services that use CENC today.  We should align the EME spec
>>>> to support it.
>>>>
>>>>
>>>>
>>>> My recommendation is that we approve and implement issue #52 now, and
>>>> review and confirm the algorithm changes in issue #41 to support them.
>>>>
>>>>
>>>>
>>>> Jerry
>>>>
>>>>
>>>>
>>>> *From:* David Dorwin [mailto:ddorwin@google.com]
>>>> *Sent:* Sunday, August 2, 2015 9:09 PM
>>>> *To:* Joe Steele <steele@adobe.com>
>>>> *Cc:* <public-html-media@w3.org> <public-html-media@w3.org>
>>>> *Subject:* Re: Resolving issues around avoiding key requests
>>>>
>>>>
>>>>
>>>> I have provided specific comments on the three issues inline. To
>>>> summarize, I believe 52 (removal of a single word) depends on 41, resolving
>>>> 41 should also consider 53, and that 53 requires a significant amount of
>>>> work to develop a consistent, cohesive, and interoperable solution.
>>>>
>>>>
>>>>
>>>> No new information has been provided that demonstrates that inclusion
>>>> of these features - in an interoperable and consistent manner - is certain
>>>> or that such resolution of the issues is trivial. Given the finite cycles
>>>> of the editors, my recommendation is to continue to focus on facilitating
>>>> interoperability for the existing feature set and resolving issues that
>>>> affect many/most authors and implementers and block LC/CR.
>>>>
>>>>
>>>>
>>>> On Thu, Jul 30, 2015 at 11:07 AM, Joe Steele <steele@adobe.com> wrote:
>>>>
>>>> There are three open issues relating to keys being provided in a manner
>>>> that may allow key requests to be avoided. I am listing them below.
>>>>
>>>> *Please review all three issues carefully and respond to this thread
>>>> with your opinions and questions.*
>>>>
>>>>
>>>> Issue 52 - Remove reference to keys in Initialization Data definition
>>>>
>>>> https://github.com/w3c/encrypted-media/issues/52
>>>>
>>>>
>>>>
>>>> The addition of keys to the list of restricted data in the initData was
>>>> made without any group input as far as I can tell. This restriction flies
>>>> in the face of the CENC specification which members of this group agreed to
>>>> support. I believe that the burden should be on the author of this list to
>>>> provide justification. In the F2F meeting the justification offered for
>>>> this restriction (as I recall) was that this would lead to an interop
>>>> problem. I disagree with this and say that this can only lead to an interop
>>>> problem if in fact those keys are used by the CDM directly, which *could*
>>>> require the algorithm changes proposed in issue 41. However it does not
>>>> *necessarily* lead to an interop problem. This edit is also required to
>>>> allow for the DRM specific key metadata specified by CENC. I believe this
>>>> issue can be resolved independent of issues 41 and 53.
>>>>
>>>>
>>>>
>>>> Some corrections:
>>>>
>>>>    - As I have repeatedly explained, adding the word "key(s)" to the
>>>>    definition was not a change in behavior; it made the definition consistent
>>>>    with the algorithms, which provide no mechanism for extracting keys from
>>>>    Initialization Data. Allowing keys (other than as a detail of the opaque
>>>>    blob to be sent to the license server) would require defining the
>>>>    extraction and processing of such keys (Issue 41).
>>>>    - Just as MSE supports a subset of MP4 streams to ensure
>>>>    compatibility with the algorithms and increase the likelihood of
>>>>    interoperability, EME supports a subset of possible CENC streams with a
>>>>    preference for openness and interoperability. In other words, EME is not a
>>>>    superset of CENC.
>>>>
>>>>
>>>>
>>>> Questions:
>>>>
>>>>    - Could you clarify why you believe that keys in initData is
>>>>    currently supported by the spec? Specifically, how/where in the application
>>>>    flow/spec are they used?
>>>>    - Regarding "this can only lead to an interop problem if in fact
>>>>    those keys are used by the CDM directly": Since “DRM specific key metadata”
>>>>    is opaque - and in Adobe’s case encrypted - I believe only the CDM /
>>>>    license server can access such keys. Am I missing something? How would such
>>>>    keys be used indirectly by the CDM or by some other entity?
>>>>
>>>>
>>>> Issue 41 - Update algorithms to reflect keys being provided in the
>>>> Initialization Data
>>>>
>>>> https://github.com/w3c/encrypted-media/issues/41
>>>>
>>>>
>>>>
>>>> The justification for this proposal is that requesting a key from a
>>>> license server is expensive. If it is unnecessary to request a key because
>>>> the keys are already available to the CDM in a manner that complies with
>>>> all other security/privacy restrictions in the spec, I believe the spec
>>>> should support not issuing a key request. It has been argued that this will
>>>> lead to interop issues. I would point out that protocol and policy
>>>> differences can already cause key requests to happen on a different
>>>> schedule between Key Systems. This is not fundamentally different and I do
>>>> not believe that this change will cause much if any additional work for
>>>> developers. *This is blocked by issue 52*.
>>>>
>>>>
>>>>
>>>> As you note under Issue 53 below, the spec does not currently allow use
>>>> of keys not present in an active session (other than keys that are part of
>>>> the DRM protocol, such as platform or individualization/provisioned keys).
>>>> Thus, there is currently no mechanism for "the keys [to] already [be]
>>>> available to the CDM." I don't oppose such a concept, but I think it
>>>> requires careful consideration and design to maintain consistency with the
>>>> rest of the spec as well as related features, such as issue 53.
>>>>
>>>>
>>>>
>>>> This is *blocking* issue 52 for the reason explained in that section
>>>> above. (In reality, they would likely be resolved together.)
>>>>
>>>>
>>>> Issue 53 - Allow for long-lived key encryption keys (aka "master" keys)
>>>> to increase performance
>>>>
>>>> https://github.com/w3c/encrypted-media/issues/53
>>>>
>>>>
>>>>
>>>> The justification for this proposal is supporting increased performance
>>>> for DRMs that can avoid additional license requests. Most if not all DRMs
>>>> already support some type of key chaining, where a key is either generated
>>>> on the client or delivered to the client and then that client key is
>>>> subsequently used for decrypting the content keys provided via key server
>>>> responses (aka licenses). This chaining today is invisible to the
>>>> application and is considered “out of scope” for this specification. I am
>>>> proposing that we bring a variety of key chaining in scope, specifically
>>>> when more than two keys are involved in the chain for performance reasons.
>>>> The main problem with the current text is that it assumes that keys
>>>> acquired during a session will go away once the session is closed, even
>>>> when there are other active sessions on the same MediaKeys object. This
>>>> would need to change to make retaining master keys across sessions
>>>> compliant with the spec. *This is blocked by issue 41*, since using a
>>>> master key would imply that a key request is not required in some cases.
>>>>
>>>>
>>>>
>>>> I have ideas for how this might work, but they involve significant
>>>> changes/additions to the spec, including new session type(s).
>>>>
>>>>
>>>>
>>>> Joe
>>>>
>>>>
>>>>
>>>
>>>
>>
>>
>
>