Re: [MSE] transport stream constraints vs Apple's HLS

Hi Michael,

Thanks for the feedback. I agree with Adrian that we shouldn't attempt to
handle all possible HLS content, but I agree with you that it would be
beneficial to improve compatibility in places that seem reasonable. We must
also keep in mind that we want to maintain compatibility with MPEG-DASH as
well so MPEG2-TS behavior will have to balance the requirements of both
specs.

I will also admit that I am not an expert on MPEG2-TS so I am also relying
on other experts like you, Bob Lund, and others to provide guidance. I've
spent "quality" time reading the MPEG2-TS spec and quickly realised that
MSE would be best served by trying to limit the myriad of optional features
allowed to the minimal useful set that makes sense for the web. I'm sure
there will be lively debate about what that set is, but the current text
was intended to facilitate that conversation.

I've included comments on your specific points below.

On Tue, Feb 5, 2013 at 7:58 AM, Adrian Bateman <adrianba@microsoft.com>wrote:

> Hi Michael,
>
> I think the group previously agreed that being able to support some HLS
> content would be good. That is, having it be possible to craft content that
> could be played using MSE and the same content be consumed by existing HLS
> players (rather than MSE being able to play *all* HLS content, which would
> be more complex).
>
> The MPEG2-TS content in the current spec is the first draft based on bug
> 17094 [1] "Define segment formats for MPEG2-TS". Bob Lund (cc'd) worked on
> the current draft text, which was incorporated into the spec for broader
> review in December. I'm sure Bob would appreciate feedback, as do we all,
> on how to improve the language. I'm afraid I'm no expert on this topic and
> so will rely on others to ensure we can meet the group's goals.
>
> I recommend you file bugs for the specific issues you raise articulating
> what the problem is and how your proposed language fixes it. That will
> allow others to comment and us to more easily include the conclusion of the
> discussions into the spec.
>
> Thanks,
>
> Adrian.
>
> [1] https://www.w3.org/Bugs/Public/show_bug.cgi?id=17094
>
> -----Original Message-----
> From: Michael Thornburgh [mailto:mthornbu@adobe.com]
> Sent: Thursday, January 31, 2013 3:55 PM
> To: public-html-media@w3.org
> Subject: [MSE] transport stream constraints vs Apple's HLS
>
> as of the current (31 Jan 2013) MSE editor's draft, there are a number of
> constraints on MPEG-2 transport streams that will make supporting Apple's
> HTTP Live Streaming (HLS) difficult.
>
> given the popularity of the iOS platform and HLS being the only
> allowed/supported streaming format there today and for the foreseeable
> future, there is a strong incentive for content publishers to use HLS.
>  once they're using HLS, it can be inconvenient (at best) or impractical
> and costly to support additional formats.  this reasoning has led folks to
> provide native HLS support in other platforms as well (such as Android),
> and i personally (and others independently) have written HLS clients in
> Flash.
>
> i think it would be beneficial to the Open Web if MSE could support this
> popular format with a minimum of impedance mis-match.
>

> (31 Jan 2013 MSE editor's draft) section 11's constraints on media
> segments and MPEG-2 TS segments specifically are problematic:
>
> * section 11 top: "Segments must start with a random access point to
> facilitate seamless splicing at the segment boundary" -- HLS does not have
> this constraint, and existing HLS content and encoders/packagers create
> segments that may not start with a RAP. this usually happens when
> encoders/packagers are configured to make each segment exactly the same
> duration, independent of where key frames fall.  i think this constraint
> should be relaxed such that each segment must contain at least one RAP, but
> not necessarily start with one.  the splice (or initial playback) could
> commence at the first RAP, which will often be the beginning of the segment
> but shouldn't have to be..
>

[acolwell] I've been thinking about loosening this restriction on media
segments in general. I've talked to several other people using MSE for live
broadcasts and they would like the "start at RAP" restriction dropped so
that they can lower startup latency. I think it makes sense to file a bug
to  change the spec text from "media segments must start with a RAP" to
"media segment must contain a RAP".


>
> * section 11.3.3 constraint #4: "Each PES packet must be comprised of one
> or more complete access units" -- this is unnecessarily restrictive and
> will exclude a lot of existing HLS content (and existing content encoders,
> including transport stream encoders supplied by Apple and others).  one of
> the tricks used in some encoders to reduce transport stream overhead is to
> allow the end of an access unit to spill into the beginning of the next PES
> packet, to avoid having to pad a last transport stream packet in the PES
> packet with wasted bytes.  reading this constraint in the most permissive
> way possible (which is probably not its intended meaning) would still not
> allow PES packets containing the last few bytes of the previous access unit
> and most (but the last few) bytes of a new access unit, since there would
> never be one complete access unit in any PES packet.  i think this
> constraint should be removed.  access unit semantics as defined in ISO/IEC
> 13818-1 should be sufficient, and any conformant transport stream parser
> must already support this PES/access unit overlap.
>

[acolwell] I'm going to need to lean on other experts to weigh in on this.
Presumably segments that conform to what MSE states are still compatible w/
existing HLS implementations. I'm assuming that these restrictions came
from the DASH spec for a reason.

I also think that we need to be careful about what we mean when we talk
about a "conformant transport stream parser". If that means supporting
everything defined in ISO/IEC 13818-1 then I'm pretty sure you'll see
pushback from UA implementors since that implies a ton of work that may not
be relevant for MSE.


>
> * section 11.3.5: "Timestamp Rollover & Discontinuities" -- in HLS,
> discontinuities are indicated in the index file, not in the transport
> stream files..  there should be a way to indicate a discontinuity via the
> API that would be interpreted identically to an in-stream discontinuity
> indication. along with this, indicating a discontinuity coincident with
> abort() must change the behavior of MPEG2TS_timestampOffset at the abort()
> (to make a contiguous splice rather than resetting MPEG2TS_timestampOffset
> to 0).
>

[acolwell] You can simulate the in-stream discontinuity by modifying
timestampOffset appropriately before appending the next segment.


> * also in section 11.3.5: in HLS, there's no provision for (or need to)
> indicate the interior media time stamps.  the same "discontinuity" API
> indication for the above could allow this to work, by having the next
> appended media snap to the expected position (like "the beginning" or
> "contiguous with previous media" or "aligned to timestampOffset").  a
> "discontinuity" API would also allow other formats to not have to know
> about their interior media timestamps, which could simplify index file
> creation in the general case.
>

[acolwell] I can see some value in being able to append a media segment
right after the previous one w/o having to know the internal timestamp of
that segment. Perhaps we could add an optional parameter to abort() that
signals "contiguous with previous media" or "aligned to timestampOffset"
like you suggest. In either case, timestampOffset would be updated to the
appropriate value once new media data is appended. You should definitely
file a bug for this.

Aaron


> -michael thornburgh
>
>
>
>
>
>
>