Addressing media files: the SMIL video element

Dear all,

I am sending this email to all your groups as requested during an SVG
meeting by Chris Lilley and Patrick Schmitz. It is believed that the
problem I will raise is of concern of all your groups.

The problem concerns the specification of the <video> element as defined
in the SVG 1.2 LC specification, taken from the SMIL specification. It
currently reads:
"The video element specifies a video file which is to be rendered to
provide synchronized video. The usual SMIL animation features are used
to start and stop the video at the appropriate times. An xlink:href is
used to link to the video content. It is assumed that the video content
also includes an audio stream, since this is the usual way that video
content is produced, and thus the audio is controlled by the video
element's media attributes."

My concern is with the last sentence in this quote. I think the video
element should only deal with video. It is a wrong to require the video
element to also render audio data. I believe video streams and audio
streams should be adressed separately.

* One argument for this is the following use case:
If a user does not want to play the audio, it would have to turn the
audio level to 0. But in this case, the player will have no way to decide:
- if the audio stream has to be decoded but not rendered. It would be
rendered only at some point in time during the presentation as a result
of an interaction, therefore decoding has to be done silently to keep
sync with the video;
- or if the audio stream can be ignored because it will never be used.

* A second argument is the following: what would happen if the the video
element adressed an AVI file with a video stream and two audio streams
(one for French, one for English) ? We should let the author decide,
possibly with feature strings testing the language. I would like to have
the opinion of the internationalisation group on this point.

One remark I had already is that the the <video> element does not
require the video file to also contain an audio track. Indeed, but it
forces an author to put all its media resources in different files if
he/she wants to play them separately, which is quite unconvenient.

My suggestion is to specify that the audio element should handle audio
streams only, and the video element video streams only. This means
removing (or deprecating) the audio related attributes from the video
element. It also means that the xlink:href attribute of those elements
have to be precise enough to identify one stream (in a container file,
on a server, ...) maybe using fragment identifiers. The mimeType
attribute in this case would describe the type of media stream and not
the type of container.

I'm looking forward to reading your comments about this problem from all
your point of views.

Best regards,
Cyril Concolato
Department COMELEC,
ENST, Paris, France
http://www.enst.fr

Received on Wednesday, 17 November 2004 13:44:51 UTC