Re: Blobs and MediaStreams (Re: getUserMedia API: How to use the LocalMediaStream object ?)

On 3/6/2012 5:03 AM, Harald Alvestrand wrote:
> Changing the subject since we're wandering far afield....
>
> On 03/06/2012 10:29 AM, Adam Bergkvist wrote:
>> On 03/05/2012 05:32 PM, Randell Jesup wrote:
>>
>>> We already have talked about the idea of navigator.getUserMedia()
>>> allowing the user to instead of giving a camera feed to replace it with
>>> a still image or pre-recorded video; modifying blob in this fashion
>>> would make that very easy to implement.
>>
>> Creating a MediaStream from a blob is interesting. We already have a
>> MediaStream constructor that takes MediaStreamTrack objects. Perhaps
>> the constructor could be overloaded to take a blob as well. Then we
>> wouldn't have to change the Blob interface.

> I'm scared of Blob/MediaStream equivalence; I think the Blob ->
> MediaStream converter is the inverse of the (so far unspecified) Record
> interface - it has the same requirement that some external knowledge
> needs to be applied in order to figure out how to decode (or encode, for
> the Recorder) the Blob.

Yes, this is an issue - how do we turn a blob (of what?) into a 
MediaStream.  There is an unstated dependence on a decoder for the blob, 
which I wasn't thinking about when I wrote that.

So we need "something" to transform blobs into mediastreams (and the 
same principle would apply for recordings, at least of data from a 
camera/mic or decoded remote data).

There are a few ways to deal with this:
  1) self-identifying streams and a 'smart' decoder
  2) out-of-band information as to the encoding (i.e. specified by the 
app directly or indirectly)

Streams on the web typically are handled by out-of-band information 
(MIME types), so any solution we make should at least support that, and 
perhaps even use that as the primary solution.

I suppose (noodling without really knowing what I'm talking about ;-) ) 
you could have a disconnected <video> or <audio> element with a blob as 
a source with a mime type supplied, and add a 'dest' or 'output' 
attribute which would be a MediaStream.  This might even make sense... 
And you'd have an obvious way to do things like provide initial scaling 
of the video, or insertion of subtitles, etc.  Hmmm.

I do want to get away from URL.createObjectURL(MediaStream) for setting 
the src for a <video> element, and I think there's a bigger opportunity 
here to unify media handling in a way that will make future work easier.

-- 
Randell Jesup
randell-ietf@jesup.org

Received on Tuesday, 6 March 2012 15:46:59 UTC