Re: Survey ready on Media Multitrack API proposal

On Wed, 03 Mar 2010 18:45:28 +0800, Silvia Pfeiffer  
<silviapfeiffer1@gmail.com> wrote:

> On Wed, Mar 3, 2010 at 5:35 PM, Michael(tm) Smith <mike@w3.org> wrote:
>> A survey is ready on the "Media Multitrack API" draft change
>> proposal.
>>
>>  http://www.w3.org/2002/09/wbs/44061/multitrack-api/
>>
>> The survey asks you to evaluate the proposal, which is for a
>> JavaScript API for extracting basic information about tracks
>> contained inside a media (audio or video) resource, and indicate
>> whether you support submitting it to the HTML WG as a decision
>> from the a11y TF; it also provides you with an opportunity to
>> suggest changes to the proposal, and, if you indicate that you
>> don't support submitting the proposal to the group, asks that you
>> provide a comment giving your reasons.
>>
>> Only members of the HTML Accessibility Task Force can complete
>> this survey. It is only a straw poll of the members of this task
>> force. However, note that the results are publicly visible:
>>
>>  http://www.w3.org/2002/09/wbs/44061/multitrack-api/results
>>
>> --
>> Michael(tm) Smith
>> http://people.w3.org/mike
>
>
> Hmm, it seems there is more discussion to be had about this proposal.
>
> I'd like to address Philip's concerns, so we can move forward.
>
>> The MediaTrack interface:
>> * Drop role unless it is actually exposed in exactly that way in  
>> existing media formats, or describe how it can be derived from the
>> information in existing media formats. Describe what value it should  
>> have when the information is not available (null or the empty
>> string?)
>
> There will be a need to map from existing media formats to this role,
> indeed. IIUC (and Eric or David can correct me), both QuickTime and
> MPEG-4 expose certain information that relates to the roles that we
> are describing. However, it is a text field and values have not been
> pre-defined, so conventions will need to be used to map to our defined
> roles.

I'll leave it to Eric to comment on the details, but it sounds like role  
should just be a free-form string, just that UAs won't attach any behavior  
to unrecognized roles.

> To be sure, Ogg does not currently require such a field, though the
> Skeleton header allows for such fields. The Xiph community is in the
> process of introducing such fields - also as a result from what we are
> finding here as necessary information to be attributed and exposed on
> a per track basis.

OK, I assume that will also be a free-form string (like Vorbis Comments)  
rather than some kind of enumeration?

>> * Type cannot be a mime type as the tracks of media files are not  
>> described with MIME types internally. For example, what is the
>> MIME type of a MP3 stream in a RIFF WAVE container? Should the MIME  
>> type actually be the type of the container, with the tracks
>> type in the codecs parameter?
>
> Mime types are actually both defined on full containers as well as
> codecs alone. That practice started with RTP/RTSP streaming when
> codecs needed to be identified rather than file types. As such, almost
> all codecs now have a mime type associated with them (e.g.
> http://www.rfc-editor.org/rfc/rfc3016.txt,
> http://tools.ietf.org/html/draft-ietf-avt-rtp-vorbis-09,
> http://tools.ietf.org/html/draft-barbato-avt-rtp-theora-01), so it
> should indeed be possible to extract/associate a mime type with a
> track.

The Vorbis/Theora ones are for RTP payload, are these appropriate for  
describing Vorbis/Theora in Ogg? They are not the same at the byte level  
if I'm not mistaken.

>> * Drop media as it belongs in the HTML markup, this information is not  
>> available from tracks in the media resource.
>
> I am not overly fussed about this one. Though I think some of the
> media information could be possible to extract about a track, e.g.
> whether it is appropriate for mobile. But seeing as we haven't really
> formalised yet what media queries really mean for media resources, it
> is a bit void to add this yet.

Thanks :)

>> Also, what about track groups? If the suggestion for referencing  
>> external tracks is going to be <trackgroup><track>, shouldn't the
>> groups of internal tracks be exposed in some way too?
>
> This is a good point. We could add a field that signifies for each
> track which track group it belongs to. Then, by looking at all tracks,
> we can identify the group. This is completely exposing the way it is
> done in MPEG.

The messy part about this is how we would represent <trackgroup><track>.  
Also, IIUC track groups are identified by integers in MPEG-4, so would  
these be exposed as is or stringified? If the track groups have meaningful  
names in themselves, perhaps there should instead be a MediaTrackGroup  
interface that has both a name and a collection of MediaTracks? I'm not  
really a fan of adding such complexity, are there other ways?

-- 
Philip Jägenstedt
Core Developer
Opera Software

Received on Wednesday, 3 March 2010 12:59:01 UTC