RE: Tech Discussions on the Multitrack Media (issue-152)

Replacement is really the only practical method of achieving something like http://www.w3.org/WAI/PF/HTML/wiki/File:BBC_iPlayer.PNG, at least until video frame manipulation through canvas becomes fast enough, or css supports quality chromakey blending of video. Using a separate rectangular video area hides to much of the source if layered over the main video, and a side by side arrangement takes up a lot of space and doesn't work for full screen video. Also the main source of audio describe video is likely to be from TV/DVD where extended descriptions are not used and pre-mixed audio is the norm. In addition, for mobile devices client side mixing of stream is a ways off. So yes I think replacement is definitely on the 80% use case side.

-----Original Message-----
From: Silvia Pfeiffer [mailto:silviapfeiffer1@gmail.com] 
Sent: 02 March 2011 21:49
To: Eric Carlson
Cc: Sean Hayes; David Singer; public-html
Subject: Re: Tech Discussions on the Multitrack Media (issue-152)

On Thu, Mar 3, 2011 at 4:22 AM, Eric Carlson <eric.carlson@apple.com> wrote:
> Hi Sean -
>
> On Mar 2, 2011, at 5:37 AM, Sean Hayes wrote:
>
>>
>> There seem to be two possible interpretations of
>>     <track kind="descriptions" srclang="en" label="English Audio Description">
>>          <source src="audesc.ogg" type="audio/ogg">
>>          <source src="audesc.mp3" type="audio/mpeg">
>>      </track>
>> One is that the audesc media is a complete replacement for the audio in the <source> element with the main soundtrack and descriptions pre-mixed, the second that the audesc is only the descriptions, and is intended to be mixed with the <source> audio at the client. Similar comments apply to the sign language option; and might even, if the captions option were of type video, apply to kind captions too where it is a replacement video with the captions burned in, or a smaller video with burned in captions to be placed PiP.
>>
>> Do you intend that the 'kind' attribute be able to distinguish between these cases, or would it be better if there were a separate '@combinator={replace|add}' attribute that allows the author to mark these cases?
>>
>  This is an interesting question but it isn't specific to our proposal, it applies to any multi-track solution.
>
>  The UA can't know if an external track augments the main resource or is a replacement for an internal track, so I think the page author must be able to signal it in the markup. I am not wild about "combinator", but I don't have a better suggestion just yet.
>
> eric


Actually, when we have replacement and the replacement resource
changes the timing of the whole virtually multitracked resource, it
makes no sense to have that as part of this construct, since it
influences all tracks. The timing of all tracks would change and thus
other resources for all tracks would need to be loaded. For this case,
a complete replacement <video> element would be much better IMHO. I am
in particular thinking about extended audio description recordings
here.

However, the case where we, e.g. want to replace an audio track with a
dubbed version in another language, or a video track with a video
track that has open captions, is much harder.  How common is this case
though? Internationalization is generally done on the Web by creating
a whole drop-in replacement site. Open captioned video should be
discouraged anyway and people should be encouraged to spend the time
re-authoring the captions in a way such that search engines and other
tools can make use of the text. Audio description tracks can be
authored without the original audio and thus be an "overlay" (i.e. an
addition).

Overall, I think that the requirements for "addition" are much more
common than for "replacement". Since replacement of resources can
already be achieved through JavaScript, I wonder if there is really a
need for inclusion of a markup solution for this. Is it really on the
80% side of the 80:20 use cases?

Cheers,
Silvia.

Received on Thursday, 3 March 2011 14:38:05 UTC