RE: In-band track metadata attribute

From: Bob Lund [mailto:B.Lund@CableLabs.com]
Sent: Wednesday, March 19, 2014 10:05 AM
To: Clift, Graham
Cc: Jon Piesing; public-inbandtracks@w3.org; Silvia Pfeiffer; Eric Winkelman; Brendan Long
Subject: Re: In-band track metadata attribute



From: <Clift>, Graham <Graham.Clift@am.sony.com<mailto:Graham.Clift@am.sony.com>>
Date: Wednesday, March 19, 2014 at 10:43 AM
To: Bob Lund <b.lund@cablelabs.com<mailto:b.lund@cablelabs.com>>
Cc: Jon Piesing <jon.piesing@philips.com<mailto:jon.piesing@philips.com>>, "public-inbandtracks@w3.org<mailto:public-inbandtracks@w3.org>" <public-inbandtracks@w3.org<mailto:public-inbandtracks@w3.org>>, Silvia Pfeiffer <silviapfeiffer1@gmail.com<mailto:silviapfeiffer1@gmail.com>>, Eric Winkelman <E.Winkelman@cablelabs.com<mailto:E.Winkelman@cablelabs.com>>, Brendan Long <B.Long@cablelabs.com<mailto:B.Long@cablelabs.com>>
Subject: Re: In-band track metadata attribute


On Mar 19, 2014 8:56 AM, Bob Lund <B.Lund@CableLabs.com<mailto:B.Lund@CableLabs.com>> wrote:
>
>
>
> From: Silvia Pfeiffer <silviapfeiffer1@gmail.com<mailto:silviapfeiffer1@gmail.com>>
> Date: Tuesday, March 18, 2014 at 12:52 AM
> To: Bob Lund <b.lund@cablelabs.com<mailto:b.lund@cablelabs.com>>
> Cc: Eric Winkelman <E.Winkelman@cablelabs.com<mailto:E.Winkelman@cablelabs.com>>, Brendan Long <B.Long@cablelabs.com<mailto:B.Long@cablelabs.com>>, "public-inbandtracks@w3.org<mailto:public-inbandtracks@w3.org>" <public-inbandtracks@w3.org<mailto:public-inbandtracks@w3.org>>
> Subject: Re: In-band track metadata attribute
> Resent-From: <public-inbandtracks@w3.org<mailto:public-inbandtracks@w3.org>>
> Resent-Date: Tuesday, March 18, 2014 at 12:53 AM
>
>> So, a PMT just describes what streams are available, similar to how the skeleton describes streams in Ogg? If so, then there is no need to expose that anywhere, since it's already inherently available to the JS developer through the tracks that are available.
>
>
> SCTE-54 and  ATSC-65 specify metadata (descriptors) that appear at the MPEG-2 TS program level (the so-called outer loop). These would not be exposed via @kind or @inBandMetadataTrackDispatchType and would require a special text track or HTMLMediaElement attribute. I am not aware of use-cases where Web applications need these descriptors but am doing more investigation. Perhaps other CG members, especially outside North America, could do the same.

Are you referring to, for example, content_advisory_descriptor and the like?
yes

Would a solution to this be to create a textTrack for the complete PMT contents as well as parsing the PMT contents to create additional tracks identified in the ES descriptor loops?
Yes, but ES descriptor info is already exposed by the inBandMeta.... Attribute and the proposed @kind expansion.
[<Graham>] I see. So the discussion is whether it makes sense to drop the PMT textTrack altogether.

The CL mapping spec original intent was to create a track for the PMT with the id set to video/mp2ts_description. Has that intent changed in the current proposals coming from CG?
A PMT track is one of the alternatives in the Wiki. It is not one of the submitted bugs. The question is are there use cases that require program level descriptors.

[<Graham>] Well I think some information might be very important like partially encrypted DTCP MPEG-2 TS streams where the (possibly expanded) CCI information is carried in a program level descriptor. See http://www.arxan.com/assets/1/7/DTCP-IP.pdf appendix.



>
>>
>> Regards,
>> Silvia.
>>
>>
>> On Tue, Mar 18, 2014 at 4:03 AM, Bob Lund <B.Lund@cablelabs.com<mailto:B.Lund@cablelabs.com>> wrote:
>>>
>>>
>>>
>>> From: Eric Winkelman <E.Winkelman@cablelabs.com<mailto:E.Winkelman@cablelabs.com>>
>>> Date: Monday, March 17, 2014 at 10:08 AM
>>> To: Brendan Long <B.Long@cablelabs.com<mailto:B.Long@cablelabs.com>>, Silvia Pfeiffer <silviapfeiffer1@gmail.com<mailto:silviapfeiffer1@gmail.com>>
>>> Cc: "public-inbandtracks@w3.org<mailto:public-inbandtracks@w3.org>" <public-inbandtracks@w3.org<mailto:public-inbandtracks@w3.org>>
>>> Subject: Re: In-band track metadata attribute
>>> Resent-From: <public-inbandtracks@w3.org<mailto:public-inbandtracks@w3.org>>
>>> Resent-Date: Monday, March 17, 2014 at 10:08 AM
>>>
>>>> On Monday, March 17, 2014 at 8:56 AM Silvia Pfeiffer <silviapfeiffer1@gmail.com<mailto:silviapfeiffer1@gmail.com>>
>>>>
>>>>>
>>>>> On 03/17/2014 03:22 AM, Silvia Pfeiffer wrote:
>>>>>>
>>>>>> On Tue, Mar 11, 2014 at 7:15 AM, Brendan Long <B.Long@cablelabs.com<mailto:B.Long@cablelabs.com>> wrote:
>>>>>>>
>>>>>>> On Tue, 2014-03-11 at 06:41 +1100, Silvia Pfeiffer wrote:
>>>>>>>>
>>>>>>>> Could you provide some examples of the data that goes there for mp2-ts?
>>>>>>>
>>>>>>> The same data that's currently listed for inBandMetadataTrackDispatchType would work, except without base64 encoding:
>>>>>>>>
>>>>>>>> Let stream type be the value of the "stream_type" field describing the text track's type in the file's program map section, interpreted as an 8-bit unsigned integer. Let length be the value of the "ES_info_length" field for the track in the same part of the program map section, interpreted as an integer as defined by the MPEG-2 specification. Let descriptor bytes be the length bytes following the "ES_info_length" field. The text track in-band metadata track dispatch type must be set to the concatenation of the stream type byte and the zero or more descriptor bytes bytes, expressed in hexadecimal using uppercase ASCII hex digits. [MPEG2]
>>>>>>>
>>>>>>> Even better might be including the entire "elementary stream info data":
>>>>>>>
>>>>>>>> Name
>>>>>>>> Number
>>>>>>>> of bits
>>>>>>>> Description
>>>>>>>> stream type
>>>>>>>> 8
>>>>>>>> This defines the structure of the data contained within the elementary packet identifier.
>>>>>>>> Reserved bits
>>>>>>>> 3
>>>>>>>> Set to 0x07 (all bits on)
>>>>>>>> Elementary PID
>>>>>>>> 13
>>>>>>>> The packet identifier that contains the stream type data.
>>>>>>>> Reserved bits
>>>>>>>> 4
>>>>>>>> Set to 0x0f (all bits on)
>>>>>>>> ES Info length unused bits
>>>>>>>> 2
>>>>>>>> Set to 0 (all bits off)
>>>>>>>> ES Info length length
>>>>>>>> 10
>>>>>>>> The number of bytes that follow for the elementary stream descriptors.
>>>>>>>> Elementary stream descriptors
>>>>>>>> N*8
>>>>>>>> When the ES info length is non-zero, this is the ES info length number of elementary stream descriptor bytes.
>>>>>>
>>>>>>
>>>>>> Is this once-off information or does it repeat? I was under the impression that, e.g. PID can repeat and change during the course of the video.
>>>>>
>>>>> The PMT appears in the file several times per second, and it can change. I think if we see a new PID, it would generally be associated with a new stream/track. I assume the PMT data for a program could change mid-stream though, but someone with more familiarity with MPEG-TS might need to confirm that. Some people we talked to suggested that changing an attribute on the track and then firing a "change" event would be easier than dealing with cues, but I'm not entirely convinced there's much difference.
>>>>
>>>>
>>>> My research into this area indicates that the PMT is allowed to change at any time, but rarely does in practice. There are lots of devices in the field that fail with frequent PMT changes, so they are avoided as much as possible.
>>>>
>>>> I agree with Brendan's suggestion that changes to the PMT result in new tracks being created. This avoids complications with trying to associate new PMT entries with the previous track (if any.) And simplifies seeking, as cues will fire in tracks with the appropriate inbandTrackMetadataDispatchType.
>>>
>>>
>>> I've queried several other experts in this area. No one has identified a use case where the PMT information for an existing stream changes. A PMT change is driven by a new/removed stream. So, from a particular track's perspective,  its PMT data is invariant.
>>>
>>>>
>>>>
>>>> Eric
>>>>
>>>>
>>