RE: [MSE] transport stream constraints vs Apple's HLS

Hi Michael,

I think the group previously agreed that being able to support some HLS content would be good. That is, having it be possible to craft content that could be played using MSE and the same content be consumed by existing HLS players (rather than MSE being able to play *all* HLS content, which would be more complex).

The MPEG2-TS content in the current spec is the first draft based on bug 17094 [1] "Define segment formats for MPEG2-TS". Bob Lund (cc'd) worked on the current draft text, which was incorporated into the spec for broader review in December. I'm sure Bob would appreciate feedback, as do we all, on how to improve the language. I'm afraid I'm no expert on this topic and so will rely on others to ensure we can meet the group's goals.

I recommend you file bugs for the specific issues you raise articulating what the problem is and how your proposed language fixes it. That will allow others to comment and us to more easily include the conclusion of the discussions into the spec.

Thanks,

Adrian.

[1] https://www.w3.org/Bugs/Public/show_bug.cgi?id=17094

-----Original Message-----
From: Michael Thornburgh [mailto:mthornbu@adobe.com] 
Sent: Thursday, January 31, 2013 3:55 PM
To: public-html-media@w3.org
Subject: [MSE] transport stream constraints vs Apple's HLS

as of the current (31 Jan 2013) MSE editor's draft, there are a number of constraints on MPEG-2 transport streams that will make supporting Apple's HTTP Live Streaming (HLS) difficult.

given the popularity of the iOS platform and HLS being the only allowed/supported streaming format there today and for the foreseeable future, there is a strong incentive for content publishers to use HLS.  once they're using HLS, it can be inconvenient (at best) or impractical and costly to support additional formats.  this reasoning has led folks to provide native HLS support in other platforms as well (such as Android), and i personally (and others independently) have written HLS clients in Flash.

i think it would be beneficial to the Open Web if MSE could support this popular format with a minimum of impedance mis-match.

(31 Jan 2013 MSE editor's draft) section 11's constraints on media segments and MPEG-2 TS segments specifically are problematic:

* section 11 top: "Segments must start with a random access point to facilitate seamless splicing at the segment boundary" -- HLS does not have this constraint, and existing HLS content and encoders/packagers create segments that may not start with a RAP. this usually happens when encoders/packagers are configured to make each segment exactly the same duration, independent of where key frames fall.  i think this constraint should be relaxed such that each segment must contain at least one RAP, but not necessarily start with one.  the splice (or initial playback) could commence at the first RAP, which will often be the beginning of the segment but shouldn't have to be..

* section 11.3.3 constraint #4: "Each PES packet must be comprised of one or more complete access units" -- this is unnecessarily restrictive and will exclude a lot of existing HLS content (and existing content encoders, including transport stream encoders supplied by Apple and others).  one of the tricks used in some encoders to reduce transport stream overhead is to allow the end of an access unit to spill into the beginning of the next PES packet, to avoid having to pad a last transport stream packet in the PES packet with wasted bytes.  reading this constraint in the most permissive way possible (which is probably not its intended meaning) would still not allow PES packets containing the last few bytes of the previous access unit and most (but the last few) bytes of a new access unit, since there would never be one complete access unit in any PES packet.  i think this constraint should be removed.  access unit semantics as defined in ISO/IEC 13818-1 should be sufficient, and any conformant transport stream parser must already support this PES/access unit overlap.

* section 11.3.5: "Timestamp Rollover & Discontinuities" -- in HLS, discontinuities are indicated in the index file, not in the transport stream files..  there should be a way to indicate a discontinuity via the API that would be interpreted identically to an in-stream discontinuity indication. along with this, indicating a discontinuity coincident with abort() must change the behavior of MPEG2TS_timestampOffset at the abort() (to make a contiguous splice rather than resetting MPEG2TS_timestampOffset to 0).

* also in section 11.3.5: in HLS, there's no provision for (or need to) indicate the interior media time stamps.  the same "discontinuity" API indication for the above could allow this to work, by having the next appended media snap to the expected position (like "the beginning" or "contiguous with previous media" or "aligned to timestampOffset").  a "discontinuity" API would also allow other formats to not have to know about their interior media timestamps, which could simplify index file creation in the general case.

-michael thornburgh

Received on Tuesday, 5 February 2013 16:03:16 UTC