Re: Blobs and MediaStreams (Re: getUserMedia API: How to use the LocalMediaStream object ?)

Adam Bergkvist wrote:
> On 03/07/2012 05:59 AM, Randell Jesup wrote:
>> On 3/6/2012 6:21 PM, Robert O'Callahan wrote:
>>>
>>> I do want to get away from URL.createObjectURL(MediaStream) for
>>> setting the src for a <video> element, and I think there's a
>>> bigger opportunity here to unify media handling in a way that will
>>> make future work easier.
>>>
>>> The MediaStreams Processing spec extends media elements' "src"
>>> attribute so you can set it directly to a MediaStream: "video.src =
>>> stream;". I've implemented that in my patches. It also adds an API to
>>> capture a MediaStream from a media element, e.g. "stream =
>>> video.captureStreamUntilEnded()".
>>
>> Right - I was forgetting the details of the Processing API proposal;
>> that's what I was really suggesting here - we should use that portion of
>> the MediaStream Processing API. To quote myself in the message that
>> kicked off this discussion:
>>
>> URLs don't help us solve other problems (and add new ones);
>> MediaStreams are a mechanism that lets us solve a whole class of
>> problems (and encapsulate utility/semantics such as audio/video
>> synchronization and multiple/alternate tracks).
>
> My objections to the "video.src = stream" approach pretty much goes away
> if it would be possible to let the src-property be of type "any" (as
> Robert suggests in the MediaStream Processing API). I'm not really sure
> what the consequences are though.

The following consequences were brought up in other recent discussions 
elsewhere:

http://lists.w3.org/Archives/Public/public-webapps/2011OctDec/1522.html

http://lists.w3.org/Archives/Public/public-webapps/2011OctDec/1516.html

These problems are solvable and some good follow up emails follow the 
links highlighted above. I think this group or the MediaStream 
Processing API account for the concerns raised in these threads 
(particularly in the case of the latter since it already extends media 
elements to accept a type of 'any').

>
> It's pretty much the src attribute as a DOMString that I see issues
> with. E.g.
>
> video1.src = stream;
> log(video1.src); // "about:MediaStream"
> video2.src = video1.src; // ??

We actually produced a snapshot spec of our first implementation here 
describing how this worked:

http://people.opera.com/richt/release/specs/device/o-device.html

To summarize, a MediaStream object that was applied to a video.src 
simply stringified to 'about:streamurl'. So in the example above 
video2.src = video1.src would simply copy the actual object assignment 
from video1 to video2 and then log(video2.src) would also simply return 
'about:streamurl'.

>>>
>>> It would also make sense to extend the "src" attribute to directly
>>> accept Blob objects as well. The behavior is just like assigning to
>>> the result of createObjectURL, except it has better GC behavior since
>>> you don't have to manually revoke the URL.

At the time of our (Opera's) initial implementation, given the challenge 
of assigning media streams to video elements, we went with the simplest 
approach possible: to assign MediaStream objects directly to video.src. 
That was before URL.createObjectURL existed.

We will be implementing URL.createObjectURL and users will be able to 
assign Stream URLs to video.src as per the current proposal. The 
question really is whether we will then remove direct stream assignment 
or keep it as a developer-friendly optimisation. That probably depends 
on working out the issues referenced in the email links I provided above 
but it's an optimization that seems within reach for us.

A number of folks from Opera have been arguing to keep this optimization 
since it seems relatively clean and straight-forward.

- Rich

Received on Wednesday, 7 March 2012 16:58:59 UTC