Re: [EME] Handling CDM availability, permissions, and supported types

it is possible that textTrack type and capability will be required as well

On Fri, Oct 3, 2014 at 4:18 PM, David Dorwin <ddorwin@google.com> wrote:

> The following is an extension of my previous proposal that addresses
> scenarios that were previously unaddressed. It uses the second Possible
> Solution I listed: Allowing the application to specify conditions to
> requestMediaKeySystem().
>
> This solution gives applications a lot of control if they want it
> (including potentially 10s of configuration combinations) while also
> supporting simple cases. Most applications will specify a list of
> configurations for each key system. This is similar to how
> isTypeSupported() may have been used except that all combinations for a
> given key system can be checked at once. Other applications might make
> multiple calls for various key systems. For example, key system A
> configuration 1 might be preferred over key system B configuration 1, which
> might be preferred over key system A configuration 2.
>
> *Spec Text*
> The following IDL and text are added:
>
> enum MediaKeysRequirement { "required", "optional", "disallowed" }
>> dictionary MediaKeySystemOptions {
>>   DOMString initDataType = "" ;
>>   DOMString audioType = "";
>>   DOMString audioCapability = "";
>>   DOMString videoType = "";
>>   DOMString videoCapability = "";
>>   MediaKeysRequirement uniqueidentifier = "optional";
>>   MediaKeysRequirement stateful = "optional";
>> };
>>
>> Non-standard additions to the this dictionary are strongly discouraged.
>> Any non-standard dictionary member MUST be of type MediaKeysRequirement.
>> This dictionary MUST NOT be used to pass state or data to the CDM.
>>
>> Note: Should dictionary members be added, it is recommended that they
>> have default values of “optional” to support the widest range of
>> application and client combinations.
>>
>>
> uniqueidentifier and stateful allow the application to check whether
> certain features are possible. This also allows the user agent to fail
> early if, for example, it is in a private browsing mode or does not support
> CDM storage.
>
>
> requestMediaKeySystem() is updated to optionally take a list of option
> dictionaries
>
> partial interface Navigator {
>>     Promise<MediaKeySystemAccess> requestMediaKeySystem(DOMString
>> keySystem*, optional Sequence<MediaKeySystemOptions>
>> supportedConfigurations*);
>> };
>
>
> |supportedConfigurations| is a list of option combinations to be tried in
> order. The algorithm would look something like this:
>
>    1. If the |keySystem| is unsupported, reject the promise.
>    2. For each element of |supportedConfigurations|:
>    1. Let options be the element.
>       2. For each dictionary member:
>          1. If the member's name is unrecognized or unhandled and the
>          value is not “optional”, continue to next iteration of the loop.
>          2. If the member’s value cannot be satisfied together in
>          combination with the previous members, continue to next iteration of the
>          loop.
>       3. If the origin is not allowed (i.e. by a user prompt) to use the
>       |keySystem| in the configuration specified by the combination of the values
>       in options, continue to next iteration of the loop.
>       4. Resolve the promise with a new MediaKeySystemAccess object.
>       (Initialization is TBD - see Areas for Discussion.)
>    3. Reject the promise. // There were no supported combinations in
>    |supportedConfigurations|.
>
>
> *Notes*
>
>    - Implementations would only prompt if all the options in the
>    dictionary can be satisfied together, avoiding unnecessarily prompting the
>    user. Implementations may additionally optimize to prompt the user in such
>    a way that a single prompt can be used to answer any combination.
>    - Audio and video capabilities are separated to address previously
>    discussed issues, such as in the "Different audio and video requirements"
>    section of https://www.w3.org/Bugs/Public/show_bug.cgi?id=24873#c0.
>    - |supportedConfigurations| is a Sequence instead of an Array because "The
>    element type of an array must not be a sequence or dictionary type.
>    <http://heycam.github.io/webidl/#idl-array>"
>       - It's unclear why this is the case or why the same rule does not
>       apply to Sequence. Maybe Boris knows.
>    - It is TBD whether rejecting the promise is the correct behavior or
>    whether the promise should instead be resolved with null. See the
>    thread beginning at
>    http://lists.w3.org/Archives/Public/public-whatwg-archive/2014Oct/0011.html
>    .
>    - This proposal addresses most issues I listed in the Concerns section
>    of my original email.
>       - New Issue #1 is explicitly addressed.
>       - Unaddressed Issue #1 could be better addressed because the user
>       agent has all possible configurations at once (unless of course the same
>       key system is checked twice, which is most likely to avoid prompts anyway).
>       - Unaddressed Issue #2 is addressed - the application can make
>       multiple requests for different options in any order.
>       - Only New Issue #2 (serialization) remains. I think that is an
>       artifact of the possibility of prompts. The only way I see to avoid this
>       would be to allow an ordered list of key systems with option arrays. This
>       seems overly complex and unnecessary, though.
>
>
> *Areas for Discussion*
>
>    - The inclusion of (a single) initDataType in the dictionary means
>    each combination may need to be duplicated (i.e. for "keyids" and "cenc").
>       - We could make this an array, but that would be inconsistent with
>       the audio/video types (i.e. all codecs must be supported).
>    - Checking multiple types requires multiple configuration entries.
>    - To address the two previous items, perhaps we should replace the
>    type-capability pairs with an array of pairs, allowing any matching pair to
>    represent a positive result. That might require defining another Dictionary
>    or other type.
>    - Depending on the outcome of the above items, we may need to add
>    methods like isTypeSupported() and isRequirementSupported() to the
>    MediaKeySystemAccess object.
>
>
> *Example Usage*
> The following assumes that non-satisfied configurations result in a
> rejected promise rather than a null value. (See the note above.) The code
> would be slightly different in the latter case.
>
> someSystemOptions = [
>  {...},
>  ...
> ];
> clearKeyOptions = [
>  {...},
>  ...
> ];
> navigator.requestMediaKeySystem("com.example.somesystem",
> someSystemOptions).then(
>   function(keySystemAccess) {
>     // Get the selected configuration in some TBD way.
>     // Use it to select the source.
>     return keySystemAccess.createMediaKeys();
>   }
> ).catch(
>   function(error) {
>     // Try the next key system.
>     navigator.requestMediaKeySystem("org.w3.clearkey",
> clearKeyOptions).then(
>     ...
>   }
> ...
> ).catch(
>   // No supported key systems .
>   ...
> );
>
> *Example Option Lists*
> The following list supports two different media formats, each in two
> different configuration (i.e. for different resolutions). Alternatively,
> the application could just check for the stream formats and adapt to what
> is available after the MediaKeySystemAccess object is returned.
>
> streamingOptions = [
>  { audioType: "audio/webm; codecs=’opus’",
>    videoType: "video/webm; codecs=’vp9’",
>    videoCapability: "foo",
>    uniqueidentifier: "required",
>  },
>  { audioType: "audio/mp4; codecs=’mp4a.XXXXXX’",
>    videoType: "video/mp4; codecs=’avc1.XXXXXX’",
>    videoCapability: "foo",
>    uniqueidentifier: "required",
>  },
>  { audioType: "audio/webm; codecs=’vorbis’",
>    videoType: "video/webm; codecs=’vp9’",
>    videoCapability: "bar",
>  },
>  { audioType: "audio/mp4; codecs=’mp4a.XXXXXX’",
>    videoType: "video/mp4; codecs=’avc1.XXXXXX’",
>    videoCapability: "bar",
>  }];
>
>
> In this scenario, an offline license requires a unique identifier and
> statefulness (to store the license).
> offlineOptions = [
>  { audioType: "audio/webm; codecs=’opus’",
>    videoType: "video/webm; codecs=’vp9’",
>    videoCapability: "foo",
>    uniqueidentifier: "required",
>    stateful: "required"
>  },
> ...
> ];
>
>
> Key release requires statefulness to store the key release receipt.
> keyReleaseOptions = [
>  { audioType: "audio/mp4; codecs=’mp4a.XXXXXX’",
>    videoType: "video/mp4; codecs=’avc1.XXXXXX’",
>    stateful: "required"
>  },
> ...
> ];
>
> On Wed, Sep 24, 2014 at 5:37 PM, David Dorwin <ddorwin@google.com> wrote:
>
>> In bug 25923 <https://www.w3.org/Bugs/Public/show_bug.cgi?id=25923>, we
>> have been discussing the proper way to report key system support when the
>> answer might depend on permissions, whether the user authorizes CDM
>> download, etc. I'd like to have a broader discussion on solving these
>> issues and get feedback from authors. I have included a concrete proposal
>> that I think is better than the current state, but it does have some
>> theoretical deficiencies.
>>
>> *Background*
>> EME currently has a synchronous isTypeSupported()
>> <https://dvcs.w3.org/hg/html-media/raw-file/default/encrypted-media/encrypted-media.html#dom-istypesupported>
>> method to determine whether key system, media type, and capability
>> combinations are supported by the user agent. Its design is similar to
>> canPlayType(). As with canPlayType(), it is expected that applications may
>> make many calls to check various combinations before picking one. The
>> current assumption is that any permission prompts would be presented during
>> the asynchronous MediaKeys.create().
>>
>> There are some scenarios that are not handled well in this model. For
>> example, how should isTypeSupported() respond if the user has not yet been
>> prompted (and thus allowed) the CDM to be used. There are other variants
>> where the CDM needs to be downloaded, which could add additional delay.
>>
>> *Proposal*
>> I propose a model inspired by Web MIDI's MIDIAccess object. Permissions,
>> installation delays, etc. would be handled as part of the creation of a new
>> object, MediaKeySystemAccess. In most cases, this object is returned
>> quickly, though asynchronously. Once the application has the object, it can
>> use it to check details (i.e. supported codecs) and instantiate the CDM
>> (creating a MediaKeys object). Since the MediaKeySystemAccess object should
>> be returned almost immediately in subsequent calls, applications should
>> still be able to use the codec checks to select streams to download just as
>> quickly (effectively) as with the current synchronous model.
>>
>> *IDL*
>> Add a new method to request access to a key system. All download and
>> permission prompts are handled here before resolving the promise. If the
>> |keySystem| is not supported (or is declined), the promise is rejected.
>> Otherwise, it is resolved with a new MediaKeySystemAccess object.
>>
>> partial interface Navigator {    Promise<MediaKeySystemAccess> requestMediaKeySystem(DOMString keySystem);
>> };
>>
>>
>> MediaKeySystemAccess contains non-static methods that were once static
>> methods in MediaKeys:
>>
>>    - isTypeSupported remains synchronous, but becomes a member and only
>>    checks the codecs and capabilities.
>>    - isInitDataTypeSupported() is added to handle exactly one attribute
>>    of the CDM. (There is no reason to check combinations of Initialization
>>    Data Types and containers/codecs.)
>>    - createMediaKeys() is still asynchronous to allow the CDM to be
>>    loaded asynchronously.
>>
>> enum IsTypeSupportedResult { "" /* empty string */, "maybe", "probably" };
>> interface MediaKeySystemAccess {
>>     boolean isInitDataTypeSupported(DOMString initDataType);
>>     IsTypeSupportedResult isTypeSupported(DOMString contentType, optional DOMString capability);
>>     Promise<MediaKeys> createMediaKeys();
>> };
>>
>>
>> MediaKeys continues to exist, but is mainly used to create
>> sessions. MediaKeySession is unchanged.
>>
>> interface MediaKeys {
>>     MediaKeySession createSession(optional SessionType sessionType = "temporary");
>>   Promise<void> setServerCertificate((ArrayBuffer or ArrayBufferView) serverCertificate);
>> };
>>
>>
>> *Example Usage*
>>
>>> navigator.requestMediaKeySystem("com.example.somesystem").then(
>>>   function(keySystemAccess) {
>>>     if (keySystemAccess.isInitDataTypeSupported("keyids") &&
>>>         (keySystemAccess.isTypeSupported("video/foo; codecs='bar,baz'")
>>> ||
>>>          keySystemAccess.isTypeSupported("video/foo;
>>> codecs='bar2,baz'"))) {
>>>       return keySystemAccess.createMediaKeys();
>>>     }
>>>     // Throw to fall into the catch.
>>>     ...
>>>   }
>>> ).catch(
>>>   function(error) {
>>>     // Try the next key system.
>>>     navigator.requestMediaKeySystem("org.w3.clearkey").then(
>>
>>     ...
>>
>>   }
>>
>> ...
>>
>> ).catch(
>>
>>   // No supported key systems .
>>
>>   ...
>>
>> );
>>
>>
>> *Concerns*
>> Although this proposal solves the scenarios mentioned in the Background
>> and that motivated bug 25923, it introduces some new ones related to
>> multiple key system support and does not address others.
>>
>> *New Issues*
>> These new issues do not exist with the current synchronous
>> isTypeSupported() model. See also the discussion beginning with comment
>> 35 <https://www.w3.org/Bugs/Public/show_bug.cgi?id=25923#c35>.
>>
>>    1. An application looking for specific type support (i.e. codecs or
>>    Initialization Data Type) may need to create multiple objects - and thus
>>    generate multiple user prompts - to find a key system that supports those
>>    type(s).
>>    2. Applications must serialize the *asynchronous* checks for multiple
>>    key systems to avoid the possibility of multiple prompts, similar to the
>>    above issue. (Note how the requestMediaKeySystem() calls are serialized in
>>    the example.)
>>
>>
>> *Unaddressed Issues*
>> These issues exist in the current model *and* various proposed models.
>>
>>    1. CDM download/install and permission to use must be presented to
>>    the user in the same UI. For example, it is not possible to download and
>>    install the CDM without also approving exposure of a unique ID (without
>>    some far-too-complex UI).
>>    2. There is no way for an application to prefer, for example, the key
>>    system that will generate the least alarming UI (or none at all). Since
>>    different CDMs (or even configurations) may have very different security
>>    and privacy properties, the related UI/UX could be different.
>>
>>
>> *Possible Solutions*
>> One or more of the above issues could potentially be addressed by the
>> following:
>>
>>    - Maintaining some level of synchronous checks for the best-case
>>    support (i.e. respond as if the CDM is installed and allowed).
>>       - Applications would still need to use the MediaKeySystemAccess
>>       methods for definitive responses.
>>    - Allowing the application to specify conditions
>>    to requestMediaKeySystem().
>>       - For example, types that must be supported or to fail if a prompt
>>       is required,
>>       - If the condition cannot be met, the promise would be rejected.
>>    - Allowing the application to query/request multiple key systems at
>>    once.
>>
>>
>
> On Wed, Sep 24, 2014 at 5:37 PM, David Dorwin <ddorwin@google.com> wrote:
>
>> In bug 25923 <https://www.w3.org/Bugs/Public/show_bug.cgi?id=25923>, we
>> have been discussing the proper way to report key system support when the
>> answer might depend on permissions, whether the user authorizes CDM
>> download, etc. I'd like to have a broader discussion on solving these
>> issues and get feedback from authors. I have included a concrete proposal
>> that I think is better than the current state, but it does have some
>> theoretical deficiencies.
>>
>> *Background*
>> EME currently has a synchronous isTypeSupported()
>> <https://dvcs.w3.org/hg/html-media/raw-file/default/encrypted-media/encrypted-media.html#dom-istypesupported>
>> method to determine whether key system, media type, and capability
>> combinations are supported by the user agent. Its design is similar to
>> canPlayType(). As with canPlayType(), it is expected that applications may
>> make many calls to check various combinations before picking one. The
>> current assumption is that any permission prompts would be presented during
>> the asynchronous MediaKeys.create().
>>
>> There are some scenarios that are not handled well in this model. For
>> example, how should isTypeSupported() respond if the user has not yet been
>> prompted (and thus allowed) the CDM to be used. There are other variants
>> where the CDM needs to be downloaded, which could add additional delay.
>>
>> *Proposal*
>> I propose a model inspired by Web MIDI's MIDIAccess object. Permissions,
>> installation delays, etc. would be handled as part of the creation of a new
>> object, MediaKeySystemAccess. In most cases, this object is returned
>> quickly, though asynchronously. Once the application has the object, it can
>> use it to check details (i.e. supported codecs) and instantiate the CDM
>> (creating a MediaKeys object). Since the MediaKeySystemAccess object should
>> be returned almost immediately in subsequent calls, applications should
>> still be able to use the codec checks to select streams to download just as
>> quickly (effectively) as with the current synchronous model.
>>
>> *IDL*
>> Add a new method to request access to a key system. All download and
>> permission prompts are handled here before resolving the promise. If the
>> |keySystem| is not supported (or is declined), the promise is rejected.
>> Otherwise, it is resolved with a new MediaKeySystemAccess object.
>>
>> partial interface Navigator {    Promise<MediaKeySystemAccess> requestMediaKeySystem(DOMString keySystem);
>> };
>>
>>
>> MediaKeySystemAccess contains non-static methods that were once static
>> methods in MediaKeys:
>>
>>    - isTypeSupported remains synchronous, but becomes a member and only
>>    checks the codecs and capabilities.
>>    - isInitDataTypeSupported() is added to handle exactly one attribute
>>    of the CDM. (There is no reason to check combinations of Initialization
>>    Data Types and containers/codecs.)
>>    - createMediaKeys() is still asynchronous to allow the CDM to be
>>    loaded asynchronously.
>>
>> enum IsTypeSupportedResult { "" /* empty string */, "maybe", "probably" };
>> interface MediaKeySystemAccess {
>>     boolean isInitDataTypeSupported(DOMString initDataType);
>>     IsTypeSupportedResult isTypeSupported(DOMString contentType, optional DOMString capability);
>>     Promise<MediaKeys> createMediaKeys();
>> };
>>
>>
>> MediaKeys continues to exist, but is mainly used to create
>> sessions. MediaKeySession is unchanged.
>>
>> interface MediaKeys {
>>     MediaKeySession createSession(optional SessionType sessionType = "temporary");
>>   Promise<void> setServerCertificate((ArrayBuffer or ArrayBufferView) serverCertificate);
>> };
>>
>>
>> *Example Usage*
>>
>>> navigator.requestMediaKeySystem("com.example.somesystem").then(
>>>   function(keySystemAccess) {
>>>     if (keySystemAccess.isInitDataTypeSupported("keyids") &&
>>>         (keySystemAccess.isTypeSupported("video/foo; codecs='bar,baz'")
>>> ||
>>>          keySystemAccess.isTypeSupported("video/foo;
>>> codecs='bar2,baz'"))) {
>>>       return keySystemAccess.createMediaKeys();
>>>     }
>>>     // Throw to fall into the catch.
>>>     ...
>>>   }
>>> ).catch(
>>>   function(error) {
>>>     // Try the next key system.
>>>     navigator.requestMediaKeySystem("org.w3.clearkey").then(
>>
>>     ...
>>
>>   }
>>
>> ...
>>
>> ).catch(
>>
>>   // No supported key systems .
>>
>>   ...
>>
>> );
>>
>>
>> *Concerns*
>> Although this proposal solves the scenarios mentioned in the Background
>> and that motivated bug 25923, it introduces some new ones related to
>> multiple key system support and does not address others.
>>
>> *New Issues*
>> These new issues do not exist with the current synchronous
>> isTypeSupported() model. See also the discussion beginning with comment
>> 35 <https://www.w3.org/Bugs/Public/show_bug.cgi?id=25923#c35>.
>>
>>    1. An application looking for specific type support (i.e. codecs or
>>    Initialization Data Type) may need to create multiple objects - and thus
>>    generate multiple user prompts - to find a key system that supports those
>>    type(s).
>>    2. Applications must serialize the *asynchronous* checks for multiple
>>    key systems to avoid the possibility of multiple prompts, similar to the
>>    above issue. (Note how the requestMediaKeySystem() calls are serialized in
>>    the example.)
>>
>>
>> *Unaddressed Issues*
>> These issues exist in the current model *and* various proposed models.
>>
>>    1. CDM download/install and permission to use must be presented to
>>    the user in the same UI. For example, it is not possible to download and
>>    install the CDM without also approving exposure of a unique ID (without
>>    some far-too-complex UI).
>>    2. There is no way for an application to prefer, for example, the key
>>    system that will generate the least alarming UI (or none at all). Since
>>    different CDMs (or even configurations) may have very different security
>>    and privacy properties, the related UI/UX could be different.
>>
>>
>> *Possible Solutions*
>> One or more of the above issues could potentially be addressed by the
>> following:
>>
>>    - Maintaining some level of synchronous checks for the best-case
>>    support (i.e. respond as if the CDM is installed and allowed).
>>       - Applications would still need to use the MediaKeySystemAccess
>>       methods for definitive responses.
>>    - Allowing the application to specify conditions
>>    to requestMediaKeySystem().
>>       - For example, types that must be supported or to fail if a prompt
>>       is required,
>>       - If the condition cannot be met, the promise would be rejected.
>>    - Allowing the application to query/request multiple key systems at
>>    once.
>>
>>
>