Re: Using WebVTT metadata tracks and missing API

On Mon, Nov 11, 2013 at 6:14 PM, Cyril Concolato
<cyril.concolato@telecom-paristech.fr> wrote:
> Le 11/11/2013 11:06, Silvia Pfeiffer a écrit :
>
>> Hi Cyril,
>>
>> On Mon, Nov 11, 2013 at 5:49 PM, Cyril Concolato
>> <cyril.concolato@telecom-paristech.fr> wrote:
>>>
>>> Le 30/10/2013 10:03, Silvia Pfeiffer a écrit :
>>>
>>>> BTW: We need to keep the connection between the video and the VTT
>>>> resource to a minimum - in particular I object to adding the mime type
>>>> of the media resource, or the codec, or codec paramters to the vtt
>>>> file header. The VTT file needs to be able to be used with different
>>>> encodings of the same video file through the <track> element, which
>>>> can relate to different <source> elements.
>>>>
>>>> So, the following header fields that you are using are actually
>>>> harmful: baseMediaFile, MPEG-4-streamType,
>>>> MPEG-4-objectTypeIndication, sampleRate, numChannels,
>>>> MPEG-4-DecoderSpecificInfo . Please don't start exposing such
>>>> information in a VTT file. This information must not be made part of a
>>>> VTT file - it is far too easy to get out of sync with an actual video
>>>> file and is already out of sync with a different encoding of the same
>>>> video file. The VTT file is not a companion file to a specific media
>>>> resource. If you need to expose such information in a file, I
>>>> recommend writing a separate file, e.g. video_metadata.xml.
>>>
>>> You misunderstood me. The information is not duplicated in the VTT file,
>>> it
>>> is *only* in the VTT file. In this example, I assume that some
>>> audio/video/other data carried in some original media container (say mp4)
>>> is
>>> not supported but the browser's demultiplexing stack and thus not exposed
>>> to
>>> JS API via a VideoTrack or AudioTrack interface. So to work around that
>>> limitation, I'm thinking about using WebVTT to expose the data (or a
>>> reference to the data). In the example, I'm putting the information that
>>> was
>>> initially in the MP4 file in the WebVTT file: some in the header (from
>>> MP4
>>> header to VTT header), some in the cue data (from MP4 samples to VTT
>>> cues).
>>
>> What is a JS developer supposed to do with this information?
>
> Parse and process it according to the format (the JS developer knows how it
> the WebVTT describes enough information) possibly using Web Audio API,
> Canvas, WebGL, SVG, ...

baseMediaFile, MPEG-4-streamType, MPEG-4-objectTypeIndication,
sampleRate, numChannels, MPEG-4-DecoderSpecificInfo all sound very
much like header fields of a media stream. You're not suggesting SW
decoding of tracks?

Silvia.

Received on Monday, 11 November 2013 10:18:49 UTC