Re: Using WebVTT metadata tracks and missing API

Le 11/11/2013 11:18, Silvia Pfeiffer a écrit :
> On Mon, Nov 11, 2013 at 6:14 PM, Cyril Concolato
> <cyril.concolato@telecom-paristech.fr> wrote:
>> Le 11/11/2013 11:06, Silvia Pfeiffer a écrit :
>>
>>> Hi Cyril,
>>>
>>> On Mon, Nov 11, 2013 at 5:49 PM, Cyril Concolato
>>> <cyril.concolato@telecom-paristech.fr> wrote:
>>>> Le 30/10/2013 10:03, Silvia Pfeiffer a écrit :
>>>>
>>>>> BTW: We need to keep the connection between the video and the VTT
>>>>> resource to a minimum - in particular I object to adding the mime type
>>>>> of the media resource, or the codec, or codec paramters to the vtt
>>>>> file header. The VTT file needs to be able to be used with different
>>>>> encodings of the same video file through the <track> element, which
>>>>> can relate to different <source> elements.
>>>>>
>>>>> So, the following header fields that you are using are actually
>>>>> harmful: baseMediaFile, MPEG-4-streamType,
>>>>> MPEG-4-objectTypeIndication, sampleRate, numChannels,
>>>>> MPEG-4-DecoderSpecificInfo . Please don't start exposing such
>>>>> information in a VTT file. This information must not be made part of a
>>>>> VTT file - it is far too easy to get out of sync with an actual video
>>>>> file and is already out of sync with a different encoding of the same
>>>>> video file. The VTT file is not a companion file to a specific media
>>>>> resource. If you need to expose such information in a file, I
>>>>> recommend writing a separate file, e.g. video_metadata.xml.
>>>> You misunderstood me. The information is not duplicated in the VTT file,
>>>> it
>>>> is *only* in the VTT file. In this example, I assume that some
>>>> audio/video/other data carried in some original media container (say mp4)
>>>> is
>>>> not supported but the browser's demultiplexing stack and thus not exposed
>>>> to
>>>> JS API via a VideoTrack or AudioTrack interface. So to work around that
>>>> limitation, I'm thinking about using WebVTT to expose the data (or a
>>>> reference to the data). In the example, I'm putting the information that
>>>> was
>>>> initially in the MP4 file in the WebVTT file: some in the header (from
>>>> MP4
>>>> header to VTT header), some in the cue data (from MP4 samples to VTT
>>>> cues).
>>> What is a JS developer supposed to do with this information?
>> Parse and process it according to the format (the JS developer knows how it
>> the WebVTT describes enough information) possibly using Web Audio API,
>> Canvas, WebGL, SVG, ...
> baseMediaFile, MPEG-4-streamType, MPEG-4-objectTypeIndication,
> sampleRate, numChannels, MPEG-4-DecoderSpecificInfo all sound very
> much like header fields of a media stream. You're not suggesting SW
> decoding of tracks?
Yes, I am. It might not make sense for all track types, on all devices, 
but for some it does (e.g. 3d graphics, motion jpeg, depth tracks, ...).

Cyril

-- 
Cyril Concolato
Maître de Conférences/Associate Professor
Groupe Multimedia/Multimedia Group
Telecom ParisTech
46 rue Barrault
75 013 Paris, France
http://concolato.wp.mines-telecom.fr/

Received on Tuesday, 12 November 2013 01:44:28 UTC