Re: About input/output channel counts

On Mon, Jun 18, 2012 at 12:41 AM, Marcus Geelnard <mage@opera.com> wrote:

> After reading the specification a few times, I'm still confused about how
> the number of input/output channels of each AudioNode is controlled/decided.
>
> Here are a few slightly contradictory indications:
>
> 1) https://dvcs.w3.org/hg/audio/**raw-file/tip/webaudio/**
> specification.html#AudioNode-**section<https://dvcs.w3.org/hg/audio/raw-file/tip/webaudio/specification.html#AudioNode-section>
>
> The AudioNode interface does not expose the number of input/output
> channels, not is any such information mentioned in the text. This gives me
> the impression that an AudioNode must process the number of channels that
> is given to it (with up-mixing applied on fan-in), and output any number of
> channels that it want. However, this does not seem to be the case for all
> nodes...
>
>
> 2) https://dvcs.w3.org/hg/audio/**raw-file/tip/webaudio/**
> specification.html#**MixerGainStructure<https://dvcs.w3.org/hg/audio/raw-file/tip/webaudio/specification.html#MixerGainStructure>
>
> This section mentions that "In cases where the channel layouts of the
> outputs do not match, an up-mix will occur to the highest number of
> channels.". Does this mean that the number of output channels of an
> AudioNode is controlled by the AudioNode itself (e.g. based on what kind of
> processing it does or possibly the number of channels in the input signal),
> regardless of what its output is connected to?
>
>
> 3) https://dvcs.w3.org/hg/audio/**raw-file/tip/webaudio/**
> specification.html#**AudioContext-section<https://dvcs.w3.org/hg/audio/raw-file/tip/webaudio/specification.html#AudioContext-section>
>
> The AudioContext.**createJavaScriptNode() method takes the arguments
> numberOfInputChannels and numberOfOutputChannels. I guess that this means
> that up-mixing/down-mixing will be applied to the input of the
> JavaScriptAudioNode to make it match the numberOfInputChannels property?
>
> Is there a similar (hidden) property for all other AudioNodes (e.g.
> DelayNode)?
>
>
> 4) https://dvcs.w3.org/hg/audio/**raw-file/tip/webaudio/**
> specification.html#**AudioGainNode<https://dvcs.w3.org/hg/audio/raw-file/tip/webaudio/specification.html#AudioGainNode>
>
> The AudioGainNode section says: "The number of channels of the output will
> always equal the number of channels of the input". In other words, the
> number of output channels is controlled by the number of input channels.
> This is different from e.g. the JavaScriptAudioNode and the AudioPannerNode.
>
>
> 5) https://dvcs.w3.org/hg/audio/**raw-file/tip/webaudio/**
> specification.html#**Convolution-reverb-effect<https://dvcs.w3.org/hg/audio/raw-file/tip/webaudio/specification.html#Convolution-reverb-effect>
>
> Figures 3 and 6 indicate that the number of output channels are NOT
> controlled by the number of channels in the convolver impulse response.
> What controls the number of outputs?
>
>
> 6) https://dvcs.w3.org/hg/audio/**raw-file/tip/webaudio/**
> specification.html#**AudioPannerNode<https://dvcs.w3.org/hg/audio/raw-file/tip/webaudio/specification.html#AudioPannerNode>
>
> The spec for the AudioPannerNode mentions: "The audio stream from the
> input will be either mono or stereo, depending on the connection(s) to the
> input."
>
> This is a bit unclear, and could for instance mean that the input signal
> will be mono for mono input, or down-mixed to stereo for inputs with >= 2
> channels?
>
> Furthermore, "The output of this node is hard-coded to stereo (2 channels)
> and currently cannot be configured.". Thus for the AudioPannerNode at
> least, the number of output channels is fixed.
>
>
> 7) https://dvcs.w3.org/hg/audio/**raw-file/tip/webaudio/**
> specification.html#**AudioChannelSplitter<https://dvcs.w3.org/hg/audio/raw-file/tip/webaudio/specification.html#AudioChannelSplitter>
>
> This section mentions: "It has a single input, and a number of "active"
> outputs which equals the number of channels in the input audio stream."
>
> This indicates that the the AudioChannelSplitter node has a flexible
> number of input channels, probably given by the number of channels of the
> connected input signal (and, I take it, with up-mixing applied in the
> fan-in case).
>
>
> .....
>
>
> So, I guess that this boils down to two question:
>
> - What controls the number of input channels of an AudioNode?
>
> - What controls the number of output channels of an AudioNode?


Hi Marcus, sorry for the confusion.  I'll definitely add all the details
for each node into the spec.

In the meantime, here's a quick summary of how the different nodes deal
with channels.  Although at first glance it might seem arbitrary, there's a
good amount of thought put into why each node type behaves as it does.  It
allows connections between nodes in sensible ways so that, for the most
part, developers don't need to worry about the channel details.

AudioBufferSourceNode
   outputs the number of channels of the buffer that it plays (its .buffer
attribute)

MediaElementAudioSourceNode
    outputs the number of channels of the <audio> or <video> element it
represents

OscillatorNode
    always outputs one channel (mono)

JavaScriptAudioNode
    Has input and output number of channels configured on creation.


AudioGainNode
DelayNode
BiquadFilterNode
WaveShaperNode
    These nodes all propagate the number of channels of the input to the
output.
    The number of channels of the input is the largest number of channels
of any of its connections.
    So, for example, if the input has both a 1-channel and 2-channel
connection, then it will be 2.

AudioPannerNode
ConvolverNode
DynamicsCompressorNode
    These have 2-channel output.

AudioChannelSplitter
    The number of channels of the input is the largest number of channels
of any of its connections.
    Each of the outputs are 1-channel.

AudioChannelMerger
    The number of channels of each input is the largest number of channels
of all connections to that input.
    The number of channels of the output is the sum total number of
channels of all inputs.