RE: [Exposed] attribute on our APIs

I realize this is bigger than the Exposed attribute, and I actually think it's not clear how things should behave (or are even implemented today).

While using the `srcObject` is the specified way to attach a MediaStream to a VideoElement, both Firefox and Chrome still support the `createObjectURL` API.
The generated URL is tied to the domain, and *today* can be used in other windows. Here is a fiddle to show this: http://jsfiddle.net/mhofman/z5ake180/
The caveat in Chrome is that the windows needs to be in the same process (one way to trigger this is window.open, or simply duplicate the tab). I'm not sure how Firefox handles its processes but it looks like the stream url is usable from all windows no matter how they're created.

Now to me this shows that there is *some* browser support to access and render a MediaStream in another window.

I realize we're late along the process for the MediaStream and Capture spec to be talking about adding structured cloning support, but here are a few things to consider:
- WebRTC is only getting now to the point where apps are moving further than the basic one-on-one video chat use case. These new use cases require more complex UX where the limit of using a single window is getting felt. Firefox Hello and Google Hangouts have a very interesting approach to having the UI outside of a regular page, but this "works" right now because calls are initiated and handled entirely using proprietary browser extensions and never involve a regular web page. If you want to transition a regular web app to such a multi-window model, we're out of luck.
- Screen sharing as a new type of media stream is barely getting stabilized as a spec. Because of that, some use cases are starting to emerge just now. Being able to pop others' video out and have them float over while sharing another tab or app is something that would be extremely useful.
- postMessage data throughput performance have historically been poor, transfer support is very limited, transferControlToProxy is nowhere to be seen, WebSockets in Workers has barely landed in Firefox, all of which has so far made usage of workers fairly impractical in the wild. However these things have been specified and are slowly getting there. At Citrix we would definitely like to access the pixels of a screen sharing stream directly in a web worker and process it there. While I realize we could get the ImageData in the main thread and send that over postMessage, this approach is inefficient at best since we incur the cost of copying a lot of large buffers.

I think we shouldn't block ourselves out of potentially interesting use cases that we are just starting to think about now, simply because it might be complex for browser to implement down the road.

While I don't want to delay LC, what's the best approach to make sure we can one day support these use cases?

Mathieu

________________________________________
From: Martin Thomson [martin.thomson@gmail.com]
Sent: Tuesday, May 19, 2015 9:29 AM
To: Harald Alvestrand
Cc: Stefan Håkansson LK; Mathieu Hofman; public-media-capture@w3.org; Iņaki Baz Castillo; annevk@annevk.nl
Subject: Re: [Exposed] attribute on our APIs

On 19 May 2015 at 03:45, Harald Alvestrand <harald@alvestrand.no> wrote:
> - There is one browser maker (Mozilla) that has said that implementing
> access to the MediaStreamTrack object in a Worker is a major engineering
> effort - so it's not cheap.

To be clear, I don't think that Workers are the problem, as long as it
is a dedicated worker.  The problem with structured clone is that it
implies postMessage, which for us could mean cross process transfer of
the media.

Received on Wednesday, 20 May 2015 19:57:45 UTC