RE: MSE questions from Open IPTV Forum, HbbTV and the UK DTG

> In addition we may have some new Bugzilla bugs filed against the MSE spec by these external experts.

I note that the Editors have responded initially to all the Bugzilla bugs filed by the external experts.  I recommend that you review these response and/or provide more information where needed directly in the appropriate bugs to keep that dialogue going.

/paulc

Paul Cotton, Microsoft Canada
17 Eleanor Drive, Ottawa, Ontario K2E 6A3
Tel: (425) 705-9596 Fax: (425) 936-7329

From: Aaron Colwell [mailto:acolwell@google.com]
Sent: Saturday, March 16, 2013 12:57 PM
To: Paul Cotton
Cc: public-html-media@w3.org; Giuseppe Pascale; Piesing, Jon
Subject: Re: MSE questions from Open IPTV Forum, HbbTV and the UK DTG

Answers to questions inline...

On Wed, Mar 13, 2013 at 10:57 AM, Paul Cotton <Paul.Cotton@microsoft.com<mailto:Paul.Cotton@microsoft.com>> wrote:

I have received the following questions from a joint meeting between the Open IPTV Forum, HbbTV and the UK DTG.    These organizations originally sent a liaison statement to the W3C Web & TV IG:

https://lists.w3.org/Archives/Member/member-web-and-tv/2013Jan/0000.html (W3C member only link)

and as a follow up to that liaison I have arranged for some of the members of these external organizations to join our Tue Mar 19 Media TF meeting.



My plan is to add these questions to the Media TF meeting agenda for Tue Mar 19.  In addition we may have some new Bugzilla bugs filed against the MSE spec by these external experts.  If such bugs are created then I will also add those to the TF meeting agenda.



Please feel free to start a discussion on these questions in response to this email.


Q1. Are there plans for test cases for MSE?

I assumed that test cases were required in later stages of the spec process. My guess is that once we get to the point of interoperability testing between browser implementations we'll share test cases with eachother. I already have a few tests in WebKit<http://trac.webkit.org/browser/trunk/LayoutTests/http/tests/media/media-source> covering Chromium's implementation, but they are not exhaustive or up to date with the current spec text yet. This set of tests will only grow as time goes on because WebKit's commit process requires tests when adding new features.




Q2. Do you believe MSE contains the features needed for a DASH player that fully supports the DASH profiles defined by MPEG and by DASH-IF? If a DASH player on top of MSE could only support a subset of these profiles, is there any information on what could not be supported?

I believe that a fair amount of DASH can be supported with MSE. I am not an expert on the DASH spec so I don't know all the details. My goal was to enable basic VOD and adaptive streaming use cases with the first version of MSE. I believe we have achieved that since I know YouTube and Netflix have both built MSE based DASH clients.


Q3. Has any thought gone into how MSE could be used with a media element whose playback rate was other than 0 or 1? Particularly rates greater than 1 or less than 0.

I would expect applications to set the playbackRate attribute on the HTMLMediaElement like non-MSE based playback would. It is up to the UA to decide how it wants to handle this. It would likely require the web application to append media segments at a much higher rate.

If applications wanted to use a keyframe only track or something, they could simply append this stream into a separate SourceBuffer and then set the selected attribute on the appropriate VideoTrack when they wanted to "fast forward". Using a separate SourceBuffer allows you to keep the keyframe only stream separate from the full frame-rate content and the application can just switch between the two video tracks using the standard track selection functionality specified in HTML5.



Q4. In section 8, it says;



>The combination of an initialization segment and any contiguous sequence of media segments associated with it must:

..

>If a track is encrypted, provide any encryption parameters necessary to

>decrypt the content (except the encryption key itself)



Is there a complete description for the handling of encrypted content with an embedded DRM?

Encryption details are handled by the Encrypted Media Extension<https://dvcs.w3.org/hg/html-media/raw-file/default/encrypted-media/encrypted-media.html> spec. They are intentionally left out of the MSE spec because implementors may want to implement MSE w/o implementing EME. The encryption information is typically inside one of the structures that reside in an initialization segment so no furthur information is needed in the MSE spec. For example, the KeyIDs for encrypted content in WebM are contained within the Tracks element which is required as part of the WebM initialization segment definition. A similar situation happens for ISO-BMFF & MPEG-2 TS.





Q5. The broadcasters in our 3 organisations have a requirement to know when a particular advert starts being played and stops being played so that they can report this information back to an advertising server. It seems to us that the (only?) way to achieve this with with MSE is to remember a) where in the timeline the advert has been inserted (as defined by timestampOffset) and b) the length of the advert. As the content is played, the app would poll the currentTime property of the media element and test if this was within an advert. Is this how you would expect this requirement to be met with MSE?

I would suggest using the metadata TextTrack<http://www.w3.org/html/wg/drafts/html/master/embedded-content-0.html#texttrack> functionality in HTML5 for this. It allows you to dynamically create Cues and add them to the timeline. You then get notified when the cue is "displayed". This will allow you to avoid polling because the event only fires when the current playback position actually reaches the cue.

Aaron