Re: Survey ready on Media Multitrack API proposal

Replies inline.


On Wed, Mar 3, 2010 at 11:58 PM, Philip Jägenstedt <philipj@opera.com> wrote:
> On Wed, 03 Mar 2010 18:45:28 +0800, Silvia Pfeiffer
> <silviapfeiffer1@gmail.com> wrote:
>
>> On Wed, Mar 3, 2010 at 5:35 PM, Michael(tm) Smith <mike@w3.org> wrote:
>>>
>>> A survey is ready on the "Media Multitrack API" draft change
>>> proposal.
>>>
>>>  http://www.w3.org/2002/09/wbs/44061/multitrack-api/
>>>
>>> The survey asks you to evaluate the proposal, which is for a
>>> JavaScript API for extracting basic information about tracks
>>> contained inside a media (audio or video) resource, and indicate
>>> whether you support submitting it to the HTML WG as a decision
>>> from the a11y TF; it also provides you with an opportunity to
>>> suggest changes to the proposal, and, if you indicate that you
>>> don't support submitting the proposal to the group, asks that you
>>> provide a comment giving your reasons.
>>>
>>> Only members of the HTML Accessibility Task Force can complete
>>> this survey. It is only a straw poll of the members of this task
>>> force. However, note that the results are publicly visible:
>>>
>>>  http://www.w3.org/2002/09/wbs/44061/multitrack-api/results
>>>
>>> --
>>> Michael(tm) Smith
>>> http://people.w3.org/mike
>>
>>
>> Hmm, it seems there is more discussion to be had about this proposal.
>>
>> I'd like to address Philip's concerns, so we can move forward.
>>
>>> The MediaTrack interface:
>>> * Drop role unless it is actually exposed in exactly that way in existing
>>> media formats, or describe how it can be derived from the
>>> information in existing media formats. Describe what value it should have
>>> when the information is not available (null or the empty
>>> string?)
>>
>> There will be a need to map from existing media formats to this role,
>> indeed. IIUC (and Eric or David can correct me), both QuickTime and
>> MPEG-4 expose certain information that relates to the roles that we
>> are describing. However, it is a text field and values have not been
>> pre-defined, so conventions will need to be used to map to our defined
>> roles.
>
> I'll leave it to Eric to comment on the details, but it sounds like role
> should just be a free-form string, just that UAs won't attach any behavior
> to unrecognized roles.

It believe it already is a free-form string, but with some default
roles defined - those roles that made sense. We had a discussion about
this at the subgroup teleconf, btw, and agreed that it makes sense to
pre-define a set of roles so as to make the obvious roles
interoperable. Do you disagree with that proposal?


>> To be sure, Ogg does not currently require such a field, though the
>> Skeleton header allows for such fields. The Xiph community is in the
>> process of introducing such fields - also as a result from what we are
>> finding here as necessary information to be attributed and exposed on
>> a per track basis.
>
> OK, I assume that will also be a free-form string (like Vorbis Comments)
> rather than some kind of enumeration?

Same as above: a free-form string with some pre-defined constants.
It's all somewhat related to how mime types are defined, though we
weren't going to create a registration body for this field (unless
there will be a huge demand and many conflicts between private roles).


>>> * Type cannot be a mime type as the tracks of media files are not
>>> described with MIME types internally. For example, what is the
>>> MIME type of a MP3 stream in a RIFF WAVE container? Should the MIME type
>>> actually be the type of the container, with the tracks
>>> type in the codecs parameter?
>>
>> Mime types are actually both defined on full containers as well as
>> codecs alone. That practice started with RTP/RTSP streaming when
>> codecs needed to be identified rather than file types. As such, almost
>> all codecs now have a mime type associated with them (e.g.
>> http://www.rfc-editor.org/rfc/rfc3016.txt,
>> http://tools.ietf.org/html/draft-ietf-avt-rtp-vorbis-09,
>> http://tools.ietf.org/html/draft-barbato-avt-rtp-theora-01), so it
>> should indeed be possible to extract/associate a mime type with a
>> track.
>
> The Vorbis/Theora ones are for RTP payload, are these appropriate for
> describing Vorbis/Theora in Ogg? They are not the same at the byte level if
> I'm not mistaken.

FAIK they are the same at the byte level. Ogg just provides packaging
and so does RTP/RTSP. There are some minor differences in where header
packets go etc, but that's a packaging detail. The core codec data is
the same.


>>> * Drop media as it belongs in the HTML markup, this information is not
>>> available from tracks in the media resource.
>>
>> I am not overly fussed about this one. Though I think some of the
>> media information could be possible to extract about a track, e.g.
>> whether it is appropriate for mobile. But seeing as we haven't really
>> formalised yet what media queries really mean for media resources, it
>> is a bit void to add this yet.
>
> Thanks :)

Well, there may be other opinions around. :-)


>>> Also, what about track groups? If the suggestion for referencing external
>>> tracks is going to be <trackgroup><track>, shouldn't the
>>> groups of internal tracks be exposed in some way too?
>>
>> This is a good point. We could add a field that signifies for each
>> track which track group it belongs to. Then, by looking at all tracks,
>> we can identify the group. This is completely exposing the way it is
>> done in MPEG.
>
> The messy part about this is how we would represent <trackgroup><track>.
> Also, IIUC track groups are identified by integers in MPEG-4, so would these
> be exposed as is or stringified? If the track groups have meaningful names
> in themselves, perhaps there should instead be a MediaTrackGroup interface
> that has both a name and a collection of MediaTracks? I'm not really a fan
> of adding such complexity, are there other ways?

Well, as I proposed, I would just add a single field to each track
which is a string (stringified number if necessary) that has the same
value as other tracks in the same group. I don't think MPEG has names
for the track groups, just identifies which tracks are alternatives of
each other. I don't actually think more complexity is needed than
that, but am looking for other opinions.

Cheers,
Silvia.

Received on Wednesday, 3 March 2010 13:12:26 UTC