Capturing streams from DOM nodes: open issues

So, there's this thing that Firefox does that we find to be a little
handy at times.  This thing involves turning a media element (<video>,
<audio>, etc...) into a MediaStream.  This function is provided by
methods on those nodes called captureStreamUntilEnded and
captureStream.

As I understand it, there are three open issues with these fine functions.

Read roc's more lucid summary here:
http://lists.w3.org/Archives/Public/public-media-capture/2014Feb/0065.html

1. They are poor and alone and have no home.

2. The getUserMedia spec, you know that one that deals with
MediaStream, doesn't do a great job of explaining what it means to
have an isolated stream (yeah, my fault).  In particular, it doesn't
deal with the interesting corner cases where this can happen for media
elements.  There was some text on that in the stream processing API
that the W3C passed over in favour of webaudio, but that never really
got discussed.  We should discuss that.

3. There's an interesting problem with MediaStreams of this nature
that we probably want to talk about.  This is especially interesting
in the context of media recording.  That is, when streams stop having
accessible content in them for any reason, what does the recorder do?
So far, I've managed to identify the following ways in which a stream
becomes suspended:

- for captureStream*, media can become isolated; the usual reason is
that the HTTP server providing content redirected to a cross origin
resource, rendering content inaccessible to the current page
- a captured media element is paused
- for captureStream (not UntilEnded), the current stream ends
- a camera or microphone is muted
- an RTCPeerConnection peer decides to stop sending for any of a vast
array of unknowable reasons
 - for captureStream, a new, inaccessible source is selected for playback
 - some other reasons I'm not sure of

Currently, we don't have great guidance here, but we should consider
whether we want to provide a way for an application to say to
MediaRecorder "no, don't record that stuff".

~~~
I have proposals for addressing these issues. I will share these at the meeting.

Hint: they will be:

1. Adopt them, probably into gUM, but I don't mind where
2. Write down something sensible, it's not rocket surgery
3. Add the widget and some events to surface these state changes (the
muted state might be reusable, but I need to make sure that there
isn't any uncomfortable overlap anywhere)

I might also propose that we add a similar function to <canvas> for
video, since I've had people ask about that.

Received on Thursday, 15 May 2014 05:40:04 UTC