Re: WebAudio feedback and suggestions

On Tue, Nov 15, 2011 at 7:45 PM, Jonathan Baudanza <jon@jonb.org> wrote:

> Hello!
>
> I'm working on porting www.beatlab.com to the WebAudio API.  Beatlab is a
> web based collaborative sound sequencer.  It is currently built with HTML
> and JavaScript with a hidden Flash component to do the audio manipulation..
>
> Thank you to everyone that has worked on this spec.  I'm looking forward
> to the day when I can move entirely off of flash.
>

Hi Jonathan, Beatlab looks very cool!  Thanks for working on this.


>
> I'd like to offer a few points of feedback and suggestions.
>
> == Cross domain issues ==
> The cross domain issues with XMLHTTPRequest are a huge hurdle for using
> WebAudio with any app that hosts sound files on a separate asset host.  I
> am currently using an IFRAME/postMessage workaround, but it's not
> sustainable and/or portable.
>

Is this something which can be addressed by CORS?


>
> Ideally, I would like to be able to pull an audio buffer out of an <audio>
> element.  MediaElementAudioSourceNode looks promising, but I don't think it
> supports this functionality
>

I'm working very hard on MediaElementAudioSourceNode, so it's coming.
 Please keep in mind that we don't escape from cross-domain issues here any
more than XMLHttpRequest.  Our security guys have already pointed this out
:)  In any case, we need to have a larger discussion to we can all better
understand the issues (including myself).


>
> == Scheduling events ==
> I could really use a something like the JavaScriptCueNode that Joe
> Berkovitz suggested here:
>
> http://lists.w3.org/Archives/Public/public-xg-audio/2010Oct/0015.html
>
> I'm currently making awkward use of setTimeout() to poll
> content.currentTime.  This goes haywire if window.alert dialog shows up.  I
> may try using a JavaScriptAudioNode to to simulate JavaScriptCueNode.
>

This isn't exactly the same as what Joe proposed, but we're looking at a
solution which is somewhat better than setTimeout():
https://bugs.webkit.org/show_bug.cgi?id=70061



>
> == noteOn semantics ==
>
> I would like to change the behavior of noteOn in the case where `when` is
> less than `currentTime`.
>
> If I call noteOn(5) at time 8, then I think my buffer should start
> playback immediately at buffer offset 3.
>

Is this for cases where you intentionally pass in a time which is already
in the past?  Or is it more for the case where you're scheduling times very
close to "now" and sometimes one will start rendering a little bit late?


>
> If a caller bothers to specify an exact time for playback, then the caller
> is more concerned with synchronization than he or she is concerned with
> playing back the entire buffer from the beginning.
>
> If a caller is more concerned with playing back the buffer from the
> beginning.  Then he or she would call noteOn(0).  I also think this might
> be better expressed as noteOn() without the 0.
>

Yes, others have suggested having the time be optional.  Strangely, it used
to be this way, but got changed a few months back.  But it can be changed
back.


>
> I realize I could accomplish this with noteGrainOn(), but I think changing
> the semantics of noteOn would help to better reflect the intention of the
> caller.
>
>