Re: EME: Rendering behavior undefined

Thanks Mark for you input on this.

I'm a little short on time right now, but it's primarily about 2.3:
"Where media rendering is not performed by the UA, for example in the case
of a hardware protected media pipeline, then the full set of HTML rendering
capabilities, for example CSS Transforms, may not be available. One likely
restriction is that video media may be constrained to appear only in
rectangular regions with sides parallel to the edges of the window and with
normal orientation."

It's not a normative section, but it implies that it's OK to just render to
a rectangle and totally ignore the normal rendering pipeline. I really
think we should get rid of it, as it is pretty much bound to be interpreted
this way.

--hans


On Mon, Jan 13, 2014 at 5:26 PM, Mark Watson <watsonm@netflix.com> wrote:

> Hans,
>
> Could you say exactly which paragraphs you think should be removed ?
> (sorry if you already said that below)
>
> Unless we say anything to the contrary, all the requirements of the HTML
> Media Element apply, including all the rendering requirements, compositing
> with other content etc. The possibility that the decoded frames may not be
> returned to the browser is not intended to change that. For that case,
> unless we say differently, the browser would need sufficient control of
> what happens to the frames to implement the standard Media Element behavior
> without actually seeing the bytes of the decoded frames.
>
> It is likely that the APIs that make decoded content available to
> Javascript are incompatible with the robustness requirements of some CDMs,
> so it's likely that these specific APIs won't work and we should say
> something about that.
>
> It is also possible that the case I describe above - where the browser
> does not see the decoded bytes but has sufficient control of how they are
> rendered / composited runs into implementation issues. I would expect to
> learn about such issues during the implementation phase. Right now, as the
> text stands, an implementation which didn't implement the correct
> behaviours would not be "interoperable" with one that did. At that point,
> we have to decide whether and how to allow such behavious. But I think it's
> premature to start discussing constraints now in advance of implementation
> experience (and specifically, when there are no implementors yet saying
> that they can't implement it without constraints).
>
> ...Mark
>
>
> On Mon, Jan 13, 2014 at 7:08 AM, Hans Schmucker <hansschmucker@gmail.com>wrote:
>
>> I came here under the same impressions that the authors of these bugs
>> were under:
>>
>> That it would not be realistic to expect anything but totally opaque
>> behavior from CDM vendors. But as you said, that didn't happen. We know
>> now, that CDM vendors WILL and DID accept some level of integration.
>>
>> The problem is that we're running around in circles here:
>> The paragraphs as they stand now go beyond mere decoding... they deal
>> with rendering and integration, so naturally any discussion about them will
>> go beyond what EME should deal with.
>>
>> So, the best way forward might be to remove those paragraphs from EME in
>> order to make it into a real decoding-only spec again and move the issues
>> over to the media spec where they belong. There we can discuss how
>> protected content in general should be treated (not just EME-protected
>> content).
>>
>> --hans
>>
>>
>>
>>
>>
>> On Mon, Jan 13, 2014 at 3:51 PM, Henri Sivonen <hsivonen@hsivonen.fi>wrote:
>>
>>> On Mon, Jan 13, 2014 at 2:45 PM, Philip Jägenstedt <philipj@opera.com>
>>> wrote:
>>> > On Mon, Jan 13, 2014 at 6:32 PM, Hans Schmucker <
>>> hansschmucker@gmail.com> wrote:
>>> >> In summary, they should be removed and at best be replaced with a note
>>> >> saying that usage of a CDM does not change the behavior of the video
>>> element
>>> >> in any way. Correct?
>>> >
>>> > Probably not entirely correct, since a video element can normally have
>>> > its pixels drawn to a canvas and its audio samples fed into the Web
>>> > Audio API. I haven't checked, but I'd be surprised if either of these
>>> > are possible using EME...
>>>
>>> The current spec text on that topic was added in response to
>>> https://www.w3.org/Bugs/Public/show_bug.cgi?id=21155 . It seems that
>>> the position of the spec is that the interaction with canvas, the Web
>>> Audio API and the like depends on the Robustness & Compliance
>>> provisions each CDM is subject to instead of depending on EME itself.
>>>
>>> --
>>> Henri Sivonen
>>> hsivonen@hsivonen.fi
>>> https://hsivonen.fi/
>>>
>>
>>
>