Re: Proposal for initial CG focus

Hi Bob, Erik,

Le 08/11/2013 20:57, Bob Lund a écrit :
> On 11/8/13 11:10 AM, "Eric Winkelman" <E.Winkelman@cablelabs.com> wrote:
>
>> On 11/8/13 10:41 AM, "Bob Lund" <B.Lund@CableLabs.com> wrote:
>>
>>> On 11/7/13 10:11 AM, "Cyril Concolato"
>>> <cyril.concolato@telecom-paristech.fr> wrote:
>>>
>>>>
>>>> I'm not sure I follow you here. I guess this has to do with the notion
>>>> of "Track Description TextTrack" in [2].
>>>>
>>>> This idea is interesting and I see where it comes from. An MPEG-2 TS
>>>> program can have a time-varying number of streams in a program and
>>>> stream changes are signaled with a PMT. So indeed, you could consider a
>>>> PMTTextTrack to signal those changes. If understand correctly, the
>>>> "Track Description TextTrack" is a generalization of that PMTTextTrack.
>>>> For MP4, this would correspond to a 'moov'-TextTrack, but in MP4 the
>>>> moov is static for the whole duration of the file and the number of
>>>> tracks does not change. I can't think at the moment of file-wide
>>>> time-varying metadata information that is not represented in the MP4
>>>> file format as a track. So that "Track Description TextTrack" would have
>>>> only 1 cue.
>>>>
>>>> My initial idea (not sure it covers every case in MPEG-2) was rather to
>>>> expose the time-varying information related to a track in the track
>>>> itself. For instance, for MPEG-2 TS, if a PMT updates the descriptor
>>>> associated to an elementary stream, for which there is already a created
>>>> TextTrack, I would create a new cue for that new information. So the cue
>>>> data for every stream would carry two types of information: signaling
>>>> and/or real data.
>>> I agree your proposal would work and it could even be applied to the
>>> MPEG-2 TS case. But, there was no texttrack attribute to hold that
>>> track's
>>> metadata. That is the reason I proposed exposing the PMT as a texttrack.
>>>
>>> I like your approach because it seems simpler. Also, the addition of the
>>> metadataTrackDispatchType to HTML5 essentially is the attribute
>>> containing
>>> the track metadata, at least for text tracks of @kind == metatdata. Maybe
>>> something similar should exist for other types of text tracks, and video
>>> and audio tracks, and be renamed as trackMetadata.
[Cyril] Why would you need to expose other tracks than metadata 
TextTracks? If you receive a stream of an unknown type, how could you 
map it to Audio/Video? I thought everything would be TextTracks.

>>
>> This approach has a problem when you seek to a new location in the A/V
>> stream, as you would need to scan through the track's cues looking for
>> these signaling cues. If there are a lot of data cues to look through,
>> this could be a performance issue.
[Cyril] Indeed, if you have time-varying track 'signaling' metadata that 
varies not so often compared to the frame rate of your video track, 
accessing that metadata will require scanning through many cues. I'm not 
sure if this is a real problem, as static data could be put in the 
inBandTrackMetadataDispatchType and time-varying data will likely be 
colocated with Random Access Points frames.

>
> If by 'signaling cues' you mean cues containing metadata about the the
> track, there aren't any. The track metadata would be held in an attribute
> of the track object - filled in when the track was created and unchanged
> after that.
[Cyril] Yes, but if that information is changing, you might have a 
problem. You could (as it is done in the MP4 FF when using multiple 
sample entry) add all time-varying track metadata to the track header, 
with some index and refer to that index from the cue, but if that 
information is really changing a lot and is not related to particular 
samples (e.g. RAP), then having a separate track for that data is 
probably a better option.

>> I think there are some related track maintenance issues as well, as we
>> need to ensure we always have a set of signaling cues covering the
>> seekable range.
> This raises a more general question about in-band track behavior when
> seeking in the media resource. In theory, the in-band track composition of
> the media resource could change. This issue exists independent of Cyril's
> idea.
>
> I guess there is also the question about what to do with any metadata
> associated with the media resource and not specific to an in-band track in
> the resource. Don't know if this situation exists.
[Cyri] Right, I don't know if this exists either.


HTH,
Cyril

-- 
Cyril Concolato
Maître de Conférences/Associate Professor
Groupe Multimedia/Multimedia Group
Telecom ParisTech
46 rue Barrault
75 013 Paris, France
http://concolato.wp.mines-telecom.fr/

Received on Monday, 11 November 2013 10:12:42 UTC