Re: Blobs and MediaStreams (Re: getUserMedia API: How to use the LocalMediaStream object ?)

On 3/7/2012 7:39 AM, Adam Bergkvist wrote:
> On 03/07/2012 05:59 AM, Randell Jesup wrote:
>> On 3/6/2012 6:21 PM, Robert O'Callahan wrote:
[snip]
>>> The MediaStreams Processing spec extends media elements' "src"
>>> attribute so you can set it directly to a MediaStream: "video.src =
>>> stream;". I've implemented that in my patches. It also adds an API to
>>> capture a MediaStream from a media element, e.g. "stream =
>>> video.captureStreamUntilEnded()".
[snip]
> My objections to the "video.src = stream" approach pretty much goes away
> if it would be possible to let the src-property be of type "any" (as
> Robert suggests in the MediaStream Processing API). I'm not really sure
> what the consequences are though.
>
> It's pretty much the src attribute as a DOMString that I see issues
> with. E.g.
>
> video1.src = stream;
> log(video1.src); // "about:MediaStream"
> video2.src = video1.src; // ??

I'll let others chime in here.  How does this work if it's video2.src = 
stream;?  Or with today's proposal:
   ms_url = URL.createObjectURL(stream);
   video1.src = ms_url;
   video2.src = ms_url;

I think it's all under-specified currently and probably invalid; if so 
then I don't see it as a ding against the direct .src version - but I'm 
no expert here.

One way that is specified and would work with the .src thing is to 
duplicate the stream, and then do video1.src = stream1; video2.src = 
stream2.  This would also work with createObjectURL().

>>> It would also make sense to extend the "src" attribute to directly
>>> accept Blob objects as well. The behavior is just like assigning to
>>> the result of createObjectURL, except it has better GC behavior since
>>> you don't have to manually revoke the URL.
>>>
>>> Then you could do, e.g.,
>>> var video = document.createElement("video");
>>> video.src = blob;
>>> var stream = video.captureStreamUntilEnded();
>>> video.play();
>>
>> Rob's suggestion is exactly what I would like to push here, and similar
>> to what I originally suggested - as mentioned above, this gives us a
>> clean way to substitute data for cameras/mics when building
>> MediaStreams; it promotes MediaStreams to being a common way to to
>> handle audio and video (and maintain sync for them), and gives us a
>> clean way to play back recorded blobs. Also, if there's any need for
>> the consumer of a MediaStream to affect the source, that could be fairly
>> easily implemented with MediaStreams or things built on top of them;
>> with URLs it may make it more painful.
>>
>> MediaStreams:
>> * Hold related audio and/or video tracks
>> * Can be created from Blobs via video elements (with Rob's
>> suggestion above), cameras/mics (getUserMedia), or remote media streams
>> (PeerConnections or from a remote URL via a video element with the
>> MediaStream Processing API)
>
> Is there a particular reason for involving a video element in this
> operation? An alternative could be:
>
> var stream = new MediaStream(blob);

Well, that's would be doable, but by using the video element you do get 
all the things that come with that in terms of APIs (resizing, 
subtitles, etc) which might otherwise require duplicating a lot of API 
into MediaStream, but maybe this isn't a big deal.  I agree it doesn't 
have to be done via video elements, but that has the advantage of being 
able to capture a stream from a URL as well.

Overall, I think it's cleaner using video elements (and in the process 
gets us some other good things and straightforward usage).  Others may 
disagree.

>> * Can be duplicated to feed content to multiple places
>> * Can be fed into media elements for playback (video.src =
>> mediastream, which we're suggesting)
>> * Can be recorded into a Blob (mediastream.record())
>> * Can be encoded and sent (via PeerConnection)
>> * Can be processed in arbitrary manners (with the MediaStream
>> Processing API)
>> * Allow for less-error-prone GCing of data by not needing to revoke
>> URLs
>> * is a floor wax and a desert topping ;-)
>
> Topping comes in two flavors - vanilla and "Local". ;)

I prefer chopped nuts on top, which requires Processing ;-)

-- 
Randell Jesup
randell-ietf@jesup.org

Received on Wednesday, 7 March 2012 15:22:19 UTC