[Bug 29188] New: Require at least one block from each audio and video track in media segment definition

https://www.w3.org/Bugs/Public/show_bug.cgi?id=29188

            Bug ID: 29188
           Summary: Require at least one block from each audio and video
                    track in media segment definition
           Product: HTML WG
           Version: unspecified
          Hardware: PC
                OS: Linux
            Status: NEW
          Severity: normal
          Priority: P2
         Component: Media Source Extensions
          Assignee: wolenetz@google.com
          Reporter: wolenetz@google.com
        QA Contact: public-html-bugzilla@w3.org
                CC: jdsmith@microsoft.com, mike@w3.org,
                    public-html-media@w3.org
  Target Milestone: ---

The coded frame processing algorithm assumes that a media segment consists of
interleaved coded frames in DTS order from each track. However, only the WebM
bytestream spec makes it clear that at least one coded frame from each audio
and video track must exist in each media segment. Without similar requirement
for other bytestreams, like ISO-BMFF, the coded frame processing algorithm
could produce undesired output for scenarios like:

With Audio/Video SourceBuffer, append in order:
media segment containing only video for time [100.....200)
media segment containing only audio for time [0.....100)

Since these are the first appends ever (or after some previous parser reset or
CFP discontinuity detection) to this SourceBuffer, the audio track's "last
decode timestamp" is not set prior to the append of the second media segment,
above. The result is there is no discontinuity detected during that append (so
all tracks are not told to need a random access point, and their last decode
timestamps are not reset.) Also, in sequence append mode, the timestampOffset
would be -100 from the first media segment, making the buffers from the
audio-only segment be clipped since its append doesn't trigger discontinuity
and group_start_timestamp update.

While these do not appear on the surface to be absolute blocking concerns,
users and implementors of the API would need to understand the behavior of CFP
with respect to single-stream appends to muxed A/V SourceBuffers.

This bug tracks the possibility of enforcing the CFP's assumption that at least
one coded frame from each audio and video track for the SourceBuffer be
included in each media segment.

It is my understanding that MSE API users that append ISO-BMFF bytestreams
currently are primarily dash.js or similar, for which single-track
SourceBuffers are the common usage, so enforcing constraints in spec and
implemnentations on multi-track SourceBuffers shouldn't regress those users.

-- 
You are receiving this mail because:
You are the QA Contact for the bug.

Received on Thursday, 8 October 2015 19:15:52 UTC