Re: Assigning MediaStream objects to HTML media elements


thanks for good input! There is some more comment towards the end of 
this mail.

On 12/13/2011 12:58 AM, Rich Tibbett wrote:
> Nine months ago we (Opera) were in the middle of some early
> experimentation work implementing an early proposal of the getUserMedia
> proposal. We found that we had to patch the then-current WHATWG<device>
> proposal with a number of additional API behaviors for when HTML media
> elements are hosting Media Stream objects.
> In our early specification we defined the term 'stream mode' for when
> such DOM elements are assigned Stream objects for the purposes of local
> media playback. We documented the behavior of the DOM APIs for HTML
> media elements operating in stream mode in [1].
> One of the key concepts we established is that MediaStream objects, when
> applied to audio/video elements, should act within a 'linear media
> timeline' model (also a defined term in [1]). Such a timeline is, for
> example, not seekable and has well-defined behavior in the case that the
> video element is paused and resumed. When a developer interfaces with
> the DOM APIs representing HTML Media elements, we define specific setter
> and getter behaviors to switch off things that don't make sense in a
> 'linear media timeline' model.
> For example, a developer cannot adjust the playbackRate of a live
> stream, captured from a user's webcam. I think we need to review and add
> these behaviors to the current specification to ensure we end up with
> consistent behavior across UAs implementing our future recommendation.
> In the course of our early prototyping and also as part of the provided
> spec [1] we also introduced the ability to assign Stream objects
> directly to the .src of a video or audio HTML element (e.g. video.src =
> stream). This was not included in the official WHATWG draft at the time
> (since we didn't have window.URL.createObjectURL at that time and no-one
> really knew how else we could do it).
> Right now in the W3C proposal developers are expected to indirectly mint
> a new temporary URL (via window.URL.createObjectURL) to assign a
> MediaStream to a video/audio element. To coincide with direct assignment
> of a Stream object to a video element, our work also defined how Stream
> objects that have been assigned to HTML media elements should be labeled
> if or when a developer attempts to resolve the src URL/URI. We settled
> on 'about:streamurl' to be a reserved, though unresolvable, about: URI,
> to indicate that the media element is displaying/playing an object that
> implements the MediaStream interface (and hence the media element is in
> stream mode and has all the behaviors of that mode as defined in [1]).
> This direct object assignment requires less code to assign Stream
> objects to video/audio elements. It works really well to date in all
> Opera implementations and we'd like to apply this behavior to the W3C
> spec pending further discussion and feedback.
> There are notably a number of things we might want to also discuss, such
> as what we should do if particular attributes (i.e. autoplay, loop) are
> included on any HTML media element declaration (but I'd assume they mean
> nothing in stream mode). The documentation in [1] should be a reasonable
> starting point for that discussion also.
> I notice that roc's MediaStream API Processing proposal [2] touches on a
> few of these aspects to some degree also, albeit in less formal prose
> right now. I guess we're all thinking along the same thing and we just
> need to define the exact terms and algorithms in our specification.
> Any feedback would be appreciated. The main question is: should we add
> such behaviors as these to our specification or not?
Personally, I have been thinking along the lines that we must add 
something like the table in [1] to explain what attributes and data of 
the media elements [3] that have a meaning for streams, and how they 
should be interpreted.

And [1] seems like a good starting point indeed.

Then, I must admit, I saw this task as something for the webrtc wg 
(cc'd) rather than for this TF.


> br/ Rich
> [1]
> [2]

Received on Tuesday, 13 December 2011 15:04:28 UTC