Re: In-band track metadata attribute

From: Eric Winkelman <E.Winkelman@cablelabs.com<mailto:E.Winkelman@cablelabs.com>>
Date: Monday, March 17, 2014 at 10:08 AM
To: Brendan Long <B.Long@cablelabs.com<mailto:B.Long@cablelabs.com>>, Silvia Pfeiffer <silviapfeiffer1@gmail.com<mailto:silviapfeiffer1@gmail.com>>
Cc: "public-inbandtracks@w3.org<mailto:public-inbandtracks@w3.org>" <public-inbandtracks@w3.org<mailto:public-inbandtracks@w3.org>>
Subject: Re: In-band track metadata attribute
Resent-From: <public-inbandtracks@w3.org<mailto:public-inbandtracks@w3.org>>
Resent-Date: Monday, March 17, 2014 at 10:08 AM

On Monday, March 17, 2014 at 8:56 AM Silvia Pfeiffer <silviapfeiffer1@gmail.com<mailto:silviapfeiffer1@gmail.com>>


On 03/17/2014 03:22 AM, Silvia Pfeiffer wrote:
On Tue, Mar 11, 2014 at 7:15 AM, Brendan Long <B.Long@cablelabs.com<mailto:B.Long@cablelabs.com>> wrote:
On Tue, 2014-03-11 at 06:41 +1100, Silvia Pfeiffer wrote:
Could you provide some examples of the data that goes there for mp2-ts?
The same data that's currently listed for inBandMetadataTrackDispatchType would work, except without base64 encoding:
Let stream type be the value of the "stream_type" field describing the text track's type in the file's program map section, interpreted as an 8-bit unsigned integer. Let length be the value of the "ES_info_length" field for the track in the same part of the program map section, interpreted as an integer as defined by the MPEG-2 specification. Let descriptor bytes be the length bytes following the "ES_info_length" field. The text track in-band metadata track dispatch type<http://www.w3.org/html/wg/drafts/html/CR/embedded-content-0.html#text-track-in-band-metadata-track-dispatch-type> must be set to the concatenation of the stream type byte and the zero or more descriptor bytes bytes, expressed in hexadecimal using uppercase ASCII hex digits<http://www.w3.org/html/wg/drafts/html/CR/infrastructure.html#uppercase-ascii-hex-digits>. [MPEG2]<http://www.w3.org/html/wg/drafts/html/CR/references.html#refsMPEG2>
Even better might be including the entire "elementary stream info data"<http://en.wikipedia.org/wiki/Program-specific_information>:

Name    Number
of bits         Description
stream type     8       This defines the structure of the data contained within the elementary packet identifier.
Reserved bits   3       Set to 0x07 (all bits on)
Elementary PID  13      The packet identifier that contains the stream type data.
Reserved bits   4       Set to 0x0f (all bits on)
ES Info length unused bits      2       Set to 0 (all bits off)
ES Info length length   10      The number of bytes that follow for the elementary stream descriptors.
Elementary stream descriptors   N*8     When the ES info length is non-zero, this is the ES info length number of elementary stream descriptor bytes.

Is this once-off information or does it repeat? I was under the impression that, e.g. PID can repeat and change during the course of the video.
The PMT appears in the file several times per second, and it can change. I think if we see a new PID, it would generally be associated with a new stream/track. I assume the PMT data for a program could change mid-stream though, but someone with more familiarity with MPEG-TS might need to confirm that. Some people we talked to suggested that changing an attribute on the track and then firing a "change" event would be easier than dealing with cues, but I'm not entirely convinced there's much difference.

My research into this area indicates that the PMT is allowed to change at any time, but rarely does in practice. There are lots of devices in the field that fail with frequent PMT changes, so they are avoided as much as possible.

I agree with Brendan’s suggestion that changes to the PMT result in new tracks being created. This avoids complications with trying to associate new PMT entries with the previous track (if any.) And simplifies seeking, as cues will fire in tracks with the appropriate inbandTrackMetadataDispatchType.

I’ve queried several other experts in this area. No one has identified a use case where the PMT information for an existing stream changes. A PMT change is driven by a new/removed stream. So, from a particular track’s perspective,  its PMT data is invariant.


Eric