Re: Requirements for Web audio APIs

On Tue, May 24, 2011 at 4:23 AM, Ian Ni-Lewis <ilewis@google.com> wrote:

> Are you intending Streams to contain both audio and video all the
> time? Isn't it more usual to demux the audio and video before
> processing?
>

For many use-cases it's convenient for video and audio to be bundled
together into the same object, and I guess that's why Ian proposed Streams
that way. (You can look at the examples in the Streams proposal and imagine
what they would be like if you had to deal with separate Streams for audio
and video.) I don't see any real downside, since any Stream can contain only
audio or only video. For any Stream processing framework it would be trivial
to add a splitter operator to separate audio and video, if that's needed
(e.g. if you want to record only audio).

Container muxing needn't get in the way of audio/video processing; you demux
from the container, realign audio and video if the container was badly
muxed, and present the data to the processor as separate audio and video
buffers. We already have to do most of that just to play media.

Rob
-- 
"Now the Bereans were of more noble character than the Thessalonians, for
they received the message with great eagerness and examined the Scriptures
every day to see if what Paul said was true." [Acts 17:11]