Re: Using WebVTT metadata tracks and missing API

Le 11/11/2013 11:06, Silvia Pfeiffer a écrit :
> Hi Cyril,
>
> On Mon, Nov 11, 2013 at 5:49 PM, Cyril Concolato
> <cyril.concolato@telecom-paristech.fr> wrote:
>> Le 30/10/2013 10:03, Silvia Pfeiffer a écrit :
>>
>>> BTW: We need to keep the connection between the video and the VTT
>>> resource to a minimum - in particular I object to adding the mime type
>>> of the media resource, or the codec, or codec paramters to the vtt
>>> file header. The VTT file needs to be able to be used with different
>>> encodings of the same video file through the <track> element, which
>>> can relate to different <source> elements.
>>>
>>> So, the following header fields that you are using are actually
>>> harmful: baseMediaFile, MPEG-4-streamType,
>>> MPEG-4-objectTypeIndication, sampleRate, numChannels,
>>> MPEG-4-DecoderSpecificInfo . Please don't start exposing such
>>> information in a VTT file. This information must not be made part of a
>>> VTT file - it is far too easy to get out of sync with an actual video
>>> file and is already out of sync with a different encoding of the same
>>> video file. The VTT file is not a companion file to a specific media
>>> resource. If you need to expose such information in a file, I
>>> recommend writing a separate file, e.g. video_metadata.xml.
>> You misunderstood me. The information is not duplicated in the VTT file, it
>> is *only* in the VTT file. In this example, I assume that some
>> audio/video/other data carried in some original media container (say mp4) is
>> not supported but the browser's demultiplexing stack and thus not exposed to
>> JS API via a VideoTrack or AudioTrack interface. So to work around that
>> limitation, I'm thinking about using WebVTT to expose the data (or a
>> reference to the data). In the example, I'm putting the information that was
>> initially in the MP4 file in the WebVTT file: some in the header (from MP4
>> header to VTT header), some in the cue data (from MP4 samples to VTT cues).
> What is a JS developer supposed to do with this information?
Parse and process it according to the format (the JS developer knows how 
it the WebVTT describes enough information) possibly using Web Audio 
API, Canvas, WebGL, SVG, ...

Cyril

-- 
Cyril Concolato
Maître de Conférences/Associate Professor
Groupe Multimedia/Multimedia Group
Telecom ParisTech
46 rue Barrault
75 013 Paris, France
http://concolato.wp.mines-telecom.fr/

Received on Monday, 11 November 2013 10:14:59 UTC