W3C home > Mailing lists > Public > public-audio@w3.org > July to September 2012

Re: DSP API proposal/draft

From: Jussi Kalliokoski <jussi.kalliokoski@gmail.com>
Date: Mon, 23 Jul 2012 16:34:15 +0300
Message-ID: <CAJhzemWLG8ZKn5VpkWGvsdQ3QvGbO+=AuoG_qHRHGheT-k-JSw@mail.gmail.com>
To: Marcus Geelnard <mage@opera.com>
Cc: public-audio@w3.org
Hey Marcus!

I'll take a deeper look at it later, but for now I have a few
 * The API seems to be designed for most parts to deal with 1D/real data.
Imaginary/vector math is very core to DSP.
 * Most methods have no way of dealing with interleaved data. This is a
problem because a lot of data out there is in interleaved format, including
PCM. A few suggestions to alleviate the problem:
   - Add input & output stride & offset to the methods.
   - Add methods for deinterleaving/interleaving data (might be useful
 * Since convolution is attached to the filter interface, it's also limited
to 1D signals, hence not very useful for images, etc.

Good work anyway!


On Mon, Jul 23, 2012 at 4:08 PM, Marcus Geelnard <mage@opera.com> wrote:

> Hello again group!
> Hope you've had a nice weekend.
> I've put together a first (not yet complete) draft specification for a DSP
> API. I would like you (the W3 WG) to have a look at it and come with both
> high level feedback (is this a good idea to begin with?) and low level
> feedback (see points below).
> Here is the spec draft (as of 2012-Jul-23):
>   http://people.opera.com/mage/**dspapi/<http://people.opera.com/mage/dspapi/>
> Quite a bit of thought has gone into it, and some benchmarking has been
> done to find out what methods are useful to include, but I'm very open for
> discussions and changes. Here is a quick summary:
> * General
>  o There are three interfaces: DSP, FFT, Filter
>  o All interfaces are supposed to live in both the main context and in
> worker contexts.
>  o Only 1D signal processing has been considered/included.
>  o Only Float32Array is supported.
>  o Points for further discussion:
>    - Interface naming (e.g. name space bloat and collisions)?
>    - FFT & Filter are only useful as objects - should they be
> interface-less / only live on the DSP interface in some way (to minimize
> global name space pollution)?
>    - I believe that 2D and 3D signal processing is currently out of scope
> and risks bloating the API. Comments?
>    - We could possibly support Float64Array too, but that'd roughly double
> the complexity (code size, testing, etc), and I don't really see any real
> use cases for it in audio. Comments?
> * The FFT interface
>  o This is probably the most obvious one, since it has been discussed
> before and it's a key building block for lots of signal processing.
>  o Points for further discussion:
>    - The exact design of the interface - comments?
> * The Filter interface
>  o This is intended to implement both IIR and FIR filters of any order
> (inspired by the Scilab/Matlab filter function [1]).
>  o It supports cross-block filtering state.
>  o Points for further discussion:
>    - Should we simplify the case of supporting filtering of several
> channels with a single Filter object? E.g. have separate History objects
> (one for each channel) that you can manage yourself - or is the
> current interface good enough (e.g. manage your own copies of history
> arrays, or have one filter per channel)?
> * The DSP interface - general
>  o Most of the methods here are quite simple, and can easily be
> implemented in JavaScript directly.
>  o Benchmarks show that for most methods, we currently get a significant
> performance gain by using native methods even for simple operations, such
> as add(), for instance.
>  o I anticipate that many of the methods in the DSP interface will be
> possible to implement with native/close-to-native performance directly in
> JS at some point in the future, but it's quite risky to make assumptions
> about that (for Audio, we need stellar performance now).
>  o Points for further discussion:
>    - Is the interface bloated?
>    - Is it too restrictive?
> * The DSP interface - Math clones
>  o The Math object clones (sin, cos, sqrt, and friends) were all put on
> the DSP object for symmetry, even though some are obviously more
> useful/critical/important than others.
> * The DSP interface - Interpolation
>  o The sampleLinear() method uses non-uniform, edge-clamping sampling
> (inspired by the Scilab/Matlab interp1 function [2]).
>  o Non-uniform sampling can be used for implementing things such as:
>    - Traditional uniform sampling.
>    - Sweeping playback rate sampling.
>    - Looped playback (supply a looping time parameter).
>    - Wave-shaper (i.e. interpolate the shape curve using the input signal).
>  o Only linear interpolation has been defined.
>  o Points for further discussion:
>    - We probably want higher order interpolation too (cubic, sinc).
>    - Lower order interpolation too? (nearest)
>    - Should we support more specialized sampling methods (e.g. uniform
> sampling) too, to further improve performance?
> Regards,
>   Marcus
> [1] http://www.mathworks.se/help/**techdoc/ref/filter.html<http://www.mathworks.se/help/techdoc/ref/filter.html>
> [2] http://help.scilab.org/docs/5.**3.3/en_US/interp1.html<http://help.scilab.org/docs/5.3.3/en_US/interp1.html>
> --
> Marcus Geelnard
> Core Graphics Developer
> Opera Software ASA
Received on Monday, 23 July 2012 13:34:47 GMT

This archive was generated by hypermail 2.2.0+W3C-0.50 : Monday, 23 July 2012 13:34:47 GMT