Re: Use-case for audio/video track mixing with ducking effect

Hello everyone,

I looked again at our most recent use cases (UC-14 and UC-15) and 
started looking at their relevant requirements.

http://www.w3.org/2011/audio/wiki/Use_Cases_and_Requirements#UC-14:_User_Control_of_Audio

http://www.w3.org/2011/audio/wiki/Use_Cases_and_Requirements#UC-15:_Video_commentary


The mapping of these use cases to our existing requirements is quite 
straightforward, but I believe we may need to capture new requirements too.


For UC-14, how do we capture the fact that the control is given to a 
local script? How would that impact the question of whether the 
processing is done by a worker or the main thread?

For UC-15, the main new requirement is that the audio track from a video 
played from the <video> element is being processed (in this case, 
ducking). We already have requirements about the variety of sources of 
audio, but do we need to explicitely add one about the ability to 
extract and process audio from video streams?

http://www.w3.org/2011/audio/wiki/Use_Cases_and_Requirements#Sources_of_audio

Thanks,
Olivier


On 12/03/2012 15:42, Olivier Thereaux wrote:
> Following this discussion, and a recent teleconference where we agreed
> it was a valuable use case, I have drafted text for UC15:
> http://www.w3.org/2011/audio/wiki/Use_Cases_and_Requirements#UC-15:_Video_commentary
>
>
> This should complete ACTION-35: Add use case for video sync, add
> requirement to work well with mediacontroller
>
> Note that I haven't added a direct mention of MediaController in the use
> case prose itself, as it is an implementation detail and should probably
> not matter as far as usage scenarios are concerned? I have made a note
> of this in here, however:
>
> http://www.w3.org/2011/audio/wiki/Use_Cases_and_Requirements#UC15_.E2.80.94_Notes
>
>
> Feedback on the use case text welcome.
> I will be populating the related requirements for this use case shortly.
>
> Cheers,
>
> Olivier
>
>
> On 02/03/2012 13:45, Olivier Thereaux wrote:
>> On 29/02/2012 23:41, Robert O'Callahan wrote:
>>> The MediaStreams Processing document had a scenario which I think isn't
>>> covered by the existing use-cases in the wiki: "Play video with
>>> processing effects mixing in out-of-band audio tracks (in sync) (e.g.
>>> mixing in an audio commentary with audio ducking)"
>>
>> Thanks for raising this, I believe you are right. We did go through the
>> "heap" of all our sources of use cases and were due to review a few open
>> questions, including this.
>>
>> See: http://www.w3.org/2011/audio/track/actions/28
>> Chris wrote: “There was a requirement to "Seamlessly switch from one
>> input stream to another (e.g. adaptive streaming)" which I think is out
>> of scope for this group. ”
>>
>>
>>
>>> A very common example of this is DVD commentary tracks.
>>
>> Indeed, and not just commentary tracks. The BBC for instance is
>> providing audio description for a number of its programmes, and the
>> ability to start the audio description track in sync with the specific
>> video timing, mix the two tracks and ideally duck the main audio track
>> when the description track is "speaking" are realistic scenario.
>>
>>
>>> A browser-oriented use-case could be: "User wants to play a video from a
>>> Web site, with a third-party commentary track downloaded from another
>>> site."
>>
>> Likewise for a number of multilingual content, where you could want to
>> keep the original sound from an interview and have the dubbing track on
>> top, with an appropriate amount of ducking.
>>
>>
>> If I may deconstruct the issue here, could we say that this use case
>> illustrates the need for:
>>
>> * Mixing sources
>> * Ducking
>> * Syncing sources/streams with other timed media and events
>>
>> Anything I forgot?
>>
>>
>> I'd love to see demos of implementing this with the web audio API, and
>> the approaches explained on the spec differences doc.
>>
>> Olivier
>>
>

-- 
Olivier Thereaux
BBC Internet Research & Future Services