Re: Using WebVTT metadata tracks and missing API

On Tue, Oct 29, 2013 at 9:50 PM, Cyril Concolato
<cyril.concolato@telecom-paristech.fr> wrote:
> Le 29/10/2013 11:20, Silvia Pfeiffer a écrit :
>
>> On Tue, Oct 29, 2013 at 8:36 PM, Cyril Concolato
>> <cyril.concolato@telecom-paristech.fr> wrote:
>>>
>>> Le 28/10/2013 22:49, Silvia Pfeiffer a écrit :
>>>
>>>> On Tue, Oct 29, 2013 at 1:04 AM, Cyril Concolato
>>>> <cyril.concolato@telecom-paristech.fr> wrote:
>>>>>
>>>>> Hi all,
>>>>>
>>>>> I recently added to MP4Box (GPAC's MP4 manipulation tool), the ability
>>>>> to
>>>>> export any MP4 track as WEBVTT metadata tracks. I thought this might be
>>>>> of
>>>>> interest to some of you (and to the new Media Resource In-Band track
>>>>> CG).
>>>>> The post is here [1]. Comments are welcome.
>>>>>
>>>>> What I've found so far doing this is that the WebVTT interfaces are
>>>>> missing:
>>>>> - an API to get non-WebVTT settings that could be added in WebVTT
>>>>> files.
>>>>
>>>> What do you mean by "non-WebVTT settings"?
>>>
>>> I mean things like:
>>> 00:00:10.000 --> 00:00:20.000 mySettings1:value1 mySettings2:value2
>>> .... data ....
>>> Of course, one could put those settings in the payload data, but that
>>> would
>>> be handy to use the VTT parser.
>>
>> Right, I see. Since the browser drops such values, what would be the
>> use case? Do you have any concrete examples?
>
> If you look at my post, you'll see for instance a boolean indicating if the
> cue payload can be random accessed or not. But there may be others. The MP4
> file format has many possible flags per sample or cue depending on the data
> in the sample. Anyway, I'm not advocating for complete mapping from MP4 to
> WebVTT. I just think that for some use cases it could be useful to have a
> standard API to access to settings which are not part of data.

Your example settings seem a prime example for what data cues are all
about. You should put them into a JSON in a cue - that will make the
parsing even simpler.


>>>>> - an API to get the header from the WebVTT file.
>>>>
>>>> Do you mean to get that in a Web browser? WebVTT has no APIs to get
>>>> things out of WebVTT files...
>>>
>>> Well it actually does, minimally. You can get the text content of each
>>> cue
>>> (.text), or the settings (cue.align, cue.size ...). Again, one can hack
>>> that
>>> and serialize the header as a dummy cue to get its content through
>>> cue.text
>>> but that's pretty ugly.
>>
>> The TextTrackCue API is not a generic WebVTT API, but just for Web
>> browsers. Thus far, the only header field that we've added to the
>> Browser API is the VTTRegions. We may add others if there's useful
>> functionality for Web devs. Since the header is not single field with
>> single functionality, I think it makes more sense to add them
>> individually.
>
> I see and I'm sympathetic to the argument that says "we don't want generic
> APIs that will bloat the Web and not be used" but having the ability to
> access header informations in a generic way could be quite simple and would
> be helpful. For instance, getting from the header the following information
> could be useful: the mime type of the data carried in cues, the encoding
> used, and other auxiliary information depending how the data was produced
> (see my post).

There are bugs in the bug tracker for some of the header fields that
you're after. Let's work on these and see what else we are missing
after that.

BTW: We need to keep the connection between the video and the VTT
resource to a minimum - in particular I object to adding the mime type
of the media resource, or the codec, or codec paramters to the vtt
file header. The VTT file needs to be able to be used with different
encodings of the same video file through the <track> element, which
can relate to different <source> elements.

So, the following header fields that you are using are actually
harmful: baseMediaFile, MPEG-4-streamType,
MPEG-4-objectTypeIndication, sampleRate, numChannels,
MPEG-4-DecoderSpecificInfo . Please don't start exposing such
information in a VTT file. This information must not be made part of a
VTT file - it is far too easy to get out of sync with an actual video
file and is already out of sync with a different encoding of the same
video file. The VTT file is not a companion file to a specific media
resource. If you need to expose such information in a file, I
recommend writing a separate file, e.g. video_metadata.xml.

HTH.

Regards,
Silvia.

Received on Wednesday, 30 October 2013 09:04:03 UTC