Re: timing model of the media resource in HTML5

On Wed, Feb 3, 2010 at 12:17 AM, Philip Jägenstedt <philipj@opera.com> wrote:
> On Tue, 02 Feb 2010 14:08:36 +0100, Silvia Pfeiffer
> <silviapfeiffer1@gmail.com> wrote:
>
>> On Wed, Feb 3, 2010 at 12:03 AM, Silvia Pfeiffer
>> <silviapfeiffer1@gmail.com> wrote:
>>>
>>> On Tue, Feb 2, 2010 at 7:19 PM, Philip Jägenstedt <philipj@opera.com>
>>> wrote:
>>>>
>>>> On Mon, 01 Feb 2010 23:30:19 +0100, Silvia Pfeiffer
>>>> <silviapfeiffer1@gmail.com> wrote:
>>>>
>>>>> On Tue, Feb 2, 2010 at 3:59 AM, Eric Carlson <eric.carlson@apple.com>
>>>>> wrote:
>>>>>>
>>>>>> On Feb 1, 2010, at 4:19 AM, Silvia Pfeiffer wrote:
>>>>>>
>>>>>> On Fri, Jan 29, 2010 at 12:39 AM, Philip Jägenstedt
>>>>>> <philipj@opera.com>
>>>>>> wrote:
>>>>>>
>>>>>> On Wed, 27 Jan 2010 12:57:51 +0100, Silvia Pfeiffer
>>>>>>
>>>>>> If we buried the track information in a javascript API, we would
>>>>>>
>>>>>> introduce an additional dependency and we would remove the ability to
>>>>>>
>>>>>> simply parse the Web page to get at such information. For example, a
>>>>>>
>>>>>> crawler would not be able to find out that there is a resource with
>>>>>>
>>>>>> captions and would probably not bother requesting the resource for its
>>>>>>
>>>>>> captions (or other text tracks).
>>>>>>
>>>>>> Surely, robots would just index the resources themselves?
>>>>>>
>>>>>> Why download binary data of indeterminate length when you can already
>>>>>> get it out of the text of the Web page? Surely, robots would prefer to
>>>>>> get that information directly out of the Webpage and not have to go
>>>>>> and download gazillions of binary media files that they have to decode
>>>>>> to get information about them.
>>>>>>
>>>>>> Right now, everybody who sees a video element in a HTML5 page simply
>>>>>> assumes that it consists of a video and a audio track and has no other
>>>>>> information in it. This is fine in the default case and in the default
>>>>>> case no extra resource description is probably necessary. But when we
>>>>>> actually do have a richer source, we need to expose that.
>>>>>>
>>>>>>  This argument leads down a very slippery slope. If it is crucial to
>>>>>> include caption information in markup for spiders, what about other
>>>>>> media
>>>>>> file metadata that a crawler might want to read - intrinsic width and
>>>>>> height, duration, encoding format, file size, bit rate, frame rate,
>>>>>> etc,
>>>>>> etc, etc? Robots may prefer to have all of this in the page do they
>>>>>> don't
>>>>>> have to load and parse the file, but I don't think it is necessary or
>>>>>> appropriate.
>>>>>
>>>>> Not quite.
>>>>>
>>>>> It is a difference if you are a web crawler that wants to collect
>>>>> captions or one that wants to collect such file metadata. For file
>>>>> metadata, you are bound to always be successful when parsing the
>>>>> header of a binary file. So, I agree there with you.
>>>>>
>>>>> But if you are only keen on captions, you are bound to often parse
>>>>> useless information if you have to download the media file header. A
>>>>> hint inside the markup that there are captions/subtitles there and
>>>>> that it is useful to parse the file - and then parse it fully - is
>>>>> very relevant.
>>>>
>>>> Even if all browser vendors should agree that this is useful and
>>>> implemented
>>>> the suggested track markup, it will only be used by authors in very rare
>>>> situations -- when they want to populate the browser's context menu
>>>> before
>>>> HAVE_METADATA. As most videos that have multiple audio/video/text tracks
>>>> won't be marked up as such in HTML, robots will still have to download
>>>> the
>>>> headers of all videos to see if they have captions. If they want to
>>>> index
>>>> the captions (not just the fact that they exist), they'll also have to
>>>> download the whole file.
>>>
>>> I still believe it's useful to expose the tracks in a media file to
>>> the browser and to automated tools without having to use javascript to
>>> get to them or having to download the media data and decode the
>>> headers.
>>>
>>> But I don't think any browser vendors will want to implement it at
>>> this stage, so I concede.
>>>
>>> Let's instead focus on getting the JavaScript API right and get to a
>>> state where we can at least make use of such multitrack media files.
>>>
>>> I have put Eric's proposal with some slight changes (replace "type"
>>> with "role" in the examples, added a "role" attribute, added a "name"
>>> attribute, added a namedItem accessor:
>>> http://www.w3.org/WAI/PF/HTML/wiki/Media_MultitrackAPI
>>>
>>> I'd say everyone should free to edit that page as they see fit, but
>>> leave a comment on the mailing list as to why the changes were
>>> necessary.
>>
>> Philip: you mentioned
>> http://www.w3.org/TR/mediaont-api-1.0/#webidl-for-api . Do you think
>> the track elements should have some of these characteristics, too, and
>> expose them?
>
> I quite like Eric's suggestion of exposing this interface on both on the
> media element and on each track. The interface isn't as good as it could be
> yet (e.g. throwing NoValue isn't going to fly, just return undefined) but I
> do think we can reuse this and implement as much of it as possible.
>
> I've already sent some feedback to the Media Annotations WG, but a lot more
> is needed if we want to use this.

I think there is potentially value in exposing the file-level metadata
to JavaScript, but to be honest it is not currently much of my worry,
since I want to focus on solving accessibility issues.

If think the only overlap with MA  and
http://www.w3.org/WAI/PF/HTML/wiki/Media_MultitrackAPI right now are
"identifier" (which I called "name") and "language" (which I used
"lang" for in good old HTML practice).

If you would prefer using the same syntax between MA and the API we've
started developing here, feel free to harmonise them.

Cheers,
Silvia.

Received on Tuesday, 2 February 2010 23:57:40 UTC