RE: tvapi-ACTION-25: Look at the timeshifting requirements and the html video element

Hi Bin,

> Do you or does anyone know the rational of current "The User Agent must not
> buffer data from a MediaStream"?

As I understand it, MediaStream comes from WebRTC, where the where the requirement is low latency real-time audio and video for conference/chat applications. So I can see that in this context you would want to show the most recent video frames.

Some background and requirements relating to MediaStream are in the MediaStream Capture Scenarios document [1]. Section 8.10, DVR Scenarios, says:

<quote>
The ability to quickly "rewind" can be useful, especially in video conference scenarios, when you may want to quickly go back and hear something you just missed. In these scenarios, you either started a capture from the beginning of the conference and you want to seek back to a specific time, or you were only streaming it (not saving it) but you allowed yourself some amount of buffer in order to review the last X minutes of video.

To support these scenarios, buffers must be introduced (because the media stream is not implicitly buffered for this scenario). In the capture scenario, as long as the UA can access previous parts of the capture (without terminating it) then this scenario could be possible.

In the streaming case, this scenario could be supported by adding a buffer directly into the media stream itself, or by capturing the media stream as previously mentioned. Given the complexities of integrating a buffer into the MediaStream proposal, using capture to accomplish this scenario is recommended.
</quote>

This means using the MediaStream Recording API [2] to do the capture, so it seems to be consistent with my previous findings, i.e., that managing timeshifted playback becomes an application-level responsibility, and presumably using Media Source Extensions to play the timeshifted media.

I wonder if there are any example WebRTC applications we could look at that support pause, resume, and rewind?

I found some discussion of pause/resume of recordings on the media capture mailing list [5, 6], but I haven't yet found any discussion regarding preventing pausing and seeking by the HTMLMediaElement rather by than the MediaStream.

>From the TV API point of view, I'd like to see the HTMLMediaElement's pause() and play() methods allow pause and resume of live TV, so that applications don't have to implement this, and also to allow seeking and playback rate adjustment within timeshifted content. But I realise there are other considerations, such as whether changes would be needed to the HTMLMediaElement's state machine model.

Chris

[1] http://w3c.github.io/mediacapture-scenarios/scenarios.html
[2] http://www.w3.org/TR/mediastream-recording/
[3] https://lists.w3.org/Archives/Public/public-tvapi/2015Mar/0015.html
[4] http://w3c.github.io/mediacapture-main/
[5] https://lists.w3.org/Archives/Public/public-media-capture/2012Dec/0133.html
[6] https://lists.w3.org/Archives/Public/public-media-capture/2012Dec/0126.html
________________________________________
From: HU, BIN [bh526r@att.com]
Sent: 16 March 2015 20:45
To: Chris Needham; TV Control API Community Group
Subject: RE: tvapi-ACTION-25: Look at the timeshifting requirements and the html  video element

Thank you Chris for looking into this.

It seems to me that a change to Media Capture and Streams to allow buffering by the User Agent makes it easier to meet our requirement.

Do you or does anyone know the rational of current "The User Agent must not buffer data from a MediaStream"?

Thanks
Bin

-----Original Message-----
From: Chris Needham [mailto:chris.needham@bbc.co.uk]
Sent: Monday, March 16, 2015 7:49 AM
To: TV Control API Community Group
Subject: tvapi-ACTION-25: Look at the timeshifting requirements and the html video element

Hi all,

An action I took from the last meeting was to investigate support for the timeshift requirements [1].

Pause and resume are supported by the HTMLMediaElement [2, 3], but when used with a MediaStream object, constraints are applied [4] that are relevant to timeshifted playback. The MediaStream spec says: "The User Agent must not buffer data from a MediaStream. When playing, the User Agent must always play the current data from the stream." This disallows pausing, and as the MediaStream is not seekable, and the playbackRate must be 1.0, it also prevents the user from fast-forwarding or rewinding through time-shifted content.

Our requirements mapping table [5] says timeshifting could be done using the MediaStream Recording API [6], but this would mean users implementing it in JavaScript rather than it being directly supported by the platform. The considerations are similar to those Paul found with regard to the recording requirements [7], e.g., use of the timeslice parameter and ondataavailable event. An application would then, I think, have to use Media Source Extensions to play the timeshifted media.

I wonder whether the MediaStream Recording API is the right approach here, or whether we should propose a change to Media Capture and Streams to allow buffering by the User Agent?

Chris

[1] http://www.w3.org/community/tvapi/wiki/Main_Page/Technical_Requirement
[2] http://www.w3.org/html/wg/drafts/html/master/semantics.html#dom-media-pause
[3] http://www.w3.org/html/wg/drafts/html/master/semantics.html#dom-media-play
[4] http://w3c.github.io/mediacapture-main/#media-element-attributes-when-playing-a-mediastream
[5] http://www.w3.org/community/tvapi/wiki/Main_Page/Requirements_Mapping
[6] http://www.w3.org/TR/mediastream-recording/
[7] https://lists.w3.org/Archives/Public/public-tvapi/2015Feb/0008.html

________________________________________
From: TV Control API Community Group Issue Tracker [sysbot+tracker@w3.org]
Sent: 17 February 2015 14:36
To: public-tvapi@w3.org
Subject: tvapi-ACTION-25: Look at the timeshifting requirements and the html video element

tvapi-ACTION-25: Look at the timeshifting requirements and the html video element

http://www.w3.org/community/tvapi/track/actions/25

Assigned to: Chris Needham

Received on Monday, 23 March 2015 08:00:25 UTC