DSP API proposal/draft

Hello again group!

Hope you've had a nice weekend.

I've put together a first (not yet complete) draft specification for a DSP  
API. I would like you (the W3 WG) to have a look at it and come with both  
high level feedback (is this a good idea to begin with?) and low level  
feedback (see points below).

Here is the spec draft (as of 2012-Jul-23):

   http://people.opera.com/mage/dspapi/

Quite a bit of thought has gone into it, and some benchmarking has been  
done to find out what methods are useful to include, but I'm very open for  
discussions and changes. Here is a quick summary:

* General
  o There are three interfaces: DSP, FFT, Filter
  o All interfaces are supposed to live in both the main context and in  
worker contexts.
  o Only 1D signal processing has been considered/included.
  o Only Float32Array is supported.
  o Points for further discussion:
    - Interface naming (e.g. name space bloat and collisions)?
    - FFT & Filter are only useful as objects - should they be  
interface-less / only live on the DSP interface in some way (to minimize  
global name space pollution)?
    - I believe that 2D and 3D signal processing is currently out of scope  
and risks bloating the API. Comments?
    - We could possibly support Float64Array too, but that'd roughly double  
the complexity (code size, testing, etc), and I don't really see any real  
use cases for it in audio. Comments?


* The FFT interface
  o This is probably the most obvious one, since it has been discussed  
before and it's a key building block for lots of signal processing.
  o Points for further discussion:
    - The exact design of the interface - comments?


* The Filter interface
  o This is intended to implement both IIR and FIR filters of any order  
(inspired by the Scilab/Matlab filter function [1]).
  o It supports cross-block filtering state.
  o Points for further discussion:
    - Should we simplify the case of supporting filtering of several  
channels with a single Filter object? E.g. have separate History objects  
(one for each channel) that you can manage yourself - or is the
current interface good enough (e.g. manage your own copies of history  
arrays, or have one filter per channel)?


* The DSP interface - general
  o Most of the methods here are quite simple, and can easily be  
implemented in JavaScript directly.
  o Benchmarks show that for most methods, we currently get a significant  
performance gain by using native methods even for simple operations, such  
as add(), for instance.
  o I anticipate that many of the methods in the DSP interface will be  
possible to implement with native/close-to-native performance directly in  
JS at some point in the future, but it's quite risky to make assumptions  
about that (for Audio, we need stellar performance now).
  o Points for further discussion:
    - Is the interface bloated?
    - Is it too restrictive?


* The DSP interface - Math clones
  o The Math object clones (sin, cos, sqrt, and friends) were all put on  
the DSP object for symmetry, even though some are obviously more  
useful/critical/important than others.


* The DSP interface - Interpolation
  o The sampleLinear() method uses non-uniform, edge-clamping sampling  
(inspired by the Scilab/Matlab interp1 function [2]).
  o Non-uniform sampling can be used for implementing things such as:
    - Traditional uniform sampling.
    - Sweeping playback rate sampling.
    - Looped playback (supply a looping time parameter).
    - Wave-shaper (i.e. interpolate the shape curve using the input signal).
  o Only linear interpolation has been defined.
  o Points for further discussion:
    - We probably want higher order interpolation too (cubic, sinc).
    - Lower order interpolation too? (nearest)
    - Should we support more specialized sampling methods (e.g. uniform  
sampling) too, to further improve performance?


Regards,

   Marcus



[1] http://www.mathworks.se/help/techdoc/ref/filter.html
[2] http://help.scilab.org/docs/5.3.3/en_US/interp1.html

-- 
Marcus Geelnard
Core Graphics Developer
Opera Software ASA

Received on Monday, 23 July 2012 13:09:23 UTC