[Bug 22134] When do multiple SourceBuffers have to be used

https://www.w3.org/Bugs/Public/show_bug.cgi?id=22134

Aaron Colwell <acolwell@google.com> changed:

           What    |Removed                     |Added
----------------------------------------------------------------------------
                 CC|                            |acolwell@google.com

--- Comment #3 from Aaron Colwell <acolwell@google.com> ---
(In reply to comment #0)
> The specification is unclear about when multiple SourceBuffers have to be
> used (i.e. when using a single SourceBuffer will fail) and when they may be
> used (i.e. using a single SourceBuffer will not fail but may not be
> recommended).

I don't believe the MSE spec should be outlining the various use cases content
authors might have. It should provide information on howthe UA behaves and what
data it accepts and what does not. I'm happy to provide guidance on the
situations you outline below, but it isn't clear to me what should be changed
in the spec since I believe the existing text provides the appropriate hints
about what to expect.

>  
> We request that this be clarified and that the clarification cover at least
> the following scenarios;
> 1. Video and audio are each delivered separately (e.g. DASH ISOBMFF content
> where each representation contains only one media component).

This would require multiple SourceBuffers because it implies multiple
initialization segments with different track types. For example, one
initialization segment would only contian an audio track and the other
initialization segment would only contain a video track. This is outlined in
the rules that apply to initialization segments in Section 11.

> 2. One video track and multiple audio tracks (e.g. multiple languages or
> accessible audio) are each delivered separately (e.g. DASH ISOBMFF content
> where each representation contains only one media component).

This would require multiple SourceBuffers for the same reasons as above.

> 3. The audio is encoded such that part of the audio it does not meet the
> requirements of section 11 of MSE to form a single logical byte stream, e.g.
> the audio in the main content is Dolby but a number of adverts are to be
> inserted where the audio is HE-AAC. There are two variations on this, 
> i) where the video during the adverts also does not meet the requirements of
> section 11 and
> ii) where the video during the adverts does meet the requirements of section
> 11.

This would require multiple SourceBuffers because codec switches are not
allowed in a single SourceBuffer. Since multiple SourceBuffers are involved it
also implies that the content for the different codecs are represented by
different AudioTrack and VideoTrack objects. The spec makes no guarantees about
seamless transitions between tracks so the content author should not assume
these track switches will be seamless. I believe Bug 22135 is intended to
continue a discussion about this particular constraint.

> 4. A new initialization segment is needed that has a different number and/or
> type of tracks and/or different track IDs.

This is explicitly forbidden in a single SourceBuffer based on the rules in
Section 11 so multiple SourceBuffers would need to be used here. I believe that
Bug 22137 is about changing this requirement so we may be able to find some
middle ground on this one. Right now though, this scenario requires multiple
SourceBuffers because the number and type of tracks are not allowed to change
within the initialization segments of a bytestream.

> 5. The byte stream format changes, e.g. from MPEG-2 transport stream to ISO
> BMFF or vice-versa.

Bytestream format changes are not allowed within a single SourceBuffer because
when an application calls addSourceBuffer() to create a SourceBuffer it needs
to specify the mimetype of the bytestream format it intends to append to the
SourceBuffer object. Obviously MPEG-2 TS data does not conform to ISOBMFF
bytestream spec rules so you can expect a changing to MPEG2 TS in a
SourceBuffer created for ISOBMFF would trigger a decode error in step 2 of the
Segment Parser Loop. If you need to support different bytestreams then you need
to use seperate SourceBuffers for that.

I hope this clarifies things.

> 
> NOTE: This issue arises from joint discussions between the Open IPTV Forum,
> HbbTV and the UK DTG. These organizations originally sent a liaison
> statement to the W3C Web & TV IG which is archived here;
> 
> https://lists.w3.org/Archives/Member/member-web-and-tv/2013Jan/0000.html
> (W3C member only link)

-- 
You are receiving this mail because:
You are the QA Contact for the bug.

Received on Tuesday, 28 May 2013 23:38:54 UTC