Re: WebAudio feedback and suggestions

Hello Jonathan!

A cool project you have there! Looking forward to seeing you port it to a
non-flash version.

On Wed, Nov 16, 2011 at 5:45 AM, Jonathan Baudanza <jon@jonb.org> wrote:

> Hello!
>
> I'm working on porting www.beatlab.com to the WebAudio API.  Beatlab is a
> web based collaborative sound sequencer.  It is currently built with HTML
> and JavaScript with a hidden Flash component to do the audio manipulation..
>
> Thank you to everyone that has worked on this spec.  I'm looking forward
> to the day when I can move entirely off of flash.
>
> I'd like to offer a few points of feedback and suggestions.
>
> == Cross domain issues ==
> The cross domain issues with XMLHTTPRequest are a huge hurdle for using
> WebAudio with any app that hosts sound files on a separate asset host.  I
> am currently using an IFRAME/postMessage workaround, but it's not
> sustainable and/or portable.
>
> Ideally, I would like to be able to pull an audio buffer out of an <audio>
> element.  MediaElementAudioSourceNode looks promising, but I don't think it
> supports this functionality


Unfortunately I think the cross-domain issues will persist, unless the
audio is served with the Access-Control-Allow headers that allow your site
access to the resource, it cannot be exposed to an Audio API, similarily to
that of the Canvas ImageData API, that will give a security error if the
canvas has any images without those headers drawn on it. This would allow
people to work around the Cross Domain XHR to get access to information
without these headers and thus is a security issue, so I don't think it
will be possible to do, and if it is, it won't be for long. :/ If you need
access to these files, and cannot control the headers, I'm afraid the
simplest way to go is to use a proxy, that either adds these headers or is
under the same domain.

== Scheduling events ==
> I could really use a something like the JavaScriptCueNode that Joe
> Berkovitz suggested here:
>
> http://lists.w3.org/Archives/Public/public-xg-audio/2010Oct/0015.html
>
> I'm currently making awkward use of setTimeout() to poll
> content.currentTime.  This goes haywire if window.alert dialog shows up.  I
> may try using a JavaScriptAudioNode to to simulate JavaScriptCueNode.
>
> == noteOn semantics ==
>
> I would like to change the behavior of noteOn in the case where `when` is
> less than `currentTime`.
>
> If I call noteOn(5) at time 8, then I think my buffer should start
> playback immediately at buffer offset 3.
>
> If a caller bothers to specify an exact time for playback, then the caller
> is more concerned with synchronization than he or she is concerned with
> playing back the entire buffer from the beginning.
>
> If a caller is more concerned with playing back the buffer from the
> beginning.  Then he or she would call noteOn(0).  I also think this might
> be better expressed as noteOn() without the 0.
>
> I realize I could accomplish this with noteGrainOn(), but I think changing
> the semantics of noteOn would help to better reflect the intention of the
> caller.
>
>
Cheers,
Jussi