Re: Capturing streams from DOM nodes: open issues

On the issue of the MediaRecorder and suspended streams, the current 
MediaRecorder spec has pause and resume methods that the app can call.  
It will also stop recording when the MediaStream "ends".  So two questions:
1) are you thinking of an API other than pause/resume for "don't record 
that stuff"?
2) under which of the circumstances that you list below would the 
MediaStream "end"?
On 5/15/2014 1:39 AM, Martin Thomson wrote:
> So, there's this thing that Firefox does that we find to be a little
> handy at times.  This thing involves turning a media element (<video>,
> <audio>, etc...) into a MediaStream.  This function is provided by
> methods on those nodes called captureStreamUntilEnded and
> captureStream.
>
> As I understand it, there are three open issues with these fine functions.
>
> Read roc's more lucid summary here:
> http://lists.w3.org/Archives/Public/public-media-capture/2014Feb/0065.html
>
> 1. They are poor and alone and have no home.
>
> 2. The getUserMedia spec, you know that one that deals with
> MediaStream, doesn't do a great job of explaining what it means to
> have an isolated stream (yeah, my fault).  In particular, it doesn't
> deal with the interesting corner cases where this can happen for media
> elements.  There was some text on that in the stream processing API
> that the W3C passed over in favour of webaudio, but that never really
> got discussed.  We should discuss that.
>
> 3. There's an interesting problem with MediaStreams of this nature
> that we probably want to talk about.  This is especially interesting
> in the context of media recording.  That is, when streams stop having
> accessible content in them for any reason, what does the recorder do?
> So far, I've managed to identify the following ways in which a stream
> becomes suspended:
>
> - for captureStream*, media can become isolated; the usual reason is
> that the HTTP server providing content redirected to a cross origin
> resource, rendering content inaccessible to the current page
> - a captured media element is paused
> - for captureStream (not UntilEnded), the current stream ends
> - a camera or microphone is muted
> - an RTCPeerConnection peer decides to stop sending for any of a vast
> array of unknowable reasons
>   - for captureStream, a new, inaccessible source is selected for playback
>   - some other reasons I'm not sure of
>
> Currently, we don't have great guidance here, but we should consider
> whether we want to provide a way for an application to say to
> MediaRecorder "no, don't record that stuff".
>
> ~~~
> I have proposals for addressing these issues. I will share these at the meeting.
>
> Hint: they will be:
>
> 1. Adopt them, probably into gUM, but I don't mind where
> 2. Write down something sensible, it's not rocket surgery
> 3. Add the widget and some events to surface these state changes (the
> muted state might be reusable, but I need to make sure that there
> isn't any uncomfortable overlap anywhere)
>
> I might also propose that we add a similar function to <canvas> for
> video, since I've had people ask about that.
>

-- 
Jim Barnett
Genesys

Received on Thursday, 15 May 2014 13:50:52 UTC