RE: AudioBufferSourceNode.buffer how to work it.

@Chris Wilson.

 

Thanks for clarification.

Can I expect one of future draft will be tried to included more explicit description about this issue.? 

According to your opinion below 4 cases should throw exception. My understanding is correct ? J

 

< case1 >
sourcenode.buffer = null;
sourcenode.start(0);
...
sourcenode.buffer = meaningfulbuffer; // Exception “It has been tried to set buffer after AudioBufferSourceNode is already started.”

 

< case2 >
sourcenode.buffer = meaningfulbuffer;
sourcenode.start(0);
...
sourcenode.buffer = null; // Exception “It has been tried to set buffer after AudioBufferSourceNode is already started.”
...
sourcenode.buffer = meaningfulbuffer; // Exception “It has been tried to set buffer after AudioBufferSourceNode is already started.”

 

< case 3 >
sourcenode.start(0);
sourcenode.buffer = meaningfulbuffer; // Exception “It has been tried to set buffer after AudioBufferSourceNode is already started.”

 

< case 5 >
sourcenode.buffer = meaningfulbuffer1;
sourcenode.start(0); // sound buffer1

sourcenode.buffer = meaningfulbuffer2; // Exception “It has been tried to set buffer after AudioBufferSourceNode is already started.”

 

Br,

KeonHo

From: Chris Wilson [mailto:cwilso@google.com] 
Sent: Saturday, March 15, 2014 6:48 AM
To: KeonHo Kim
Cc: Raymond Toy; public-audio@w3.org
Subject: Re: AudioBufferSourceNode.buffer how to work it.

 

"null" refers to the Javascript null object; perhaps we need to be more explicit about that.

 

I have no idea why any implementation would re-start the buffer once it's already playing; personally, I think we should explicitly ignore setting the buffer once it has been set.

 

On Wed, Mar 12, 2014 at 6:51 PM, KeonHo Kim <keonho07.kim@samsung.com> wrote:





2014-03-13 3:01 GMT+09:00 Raymond Toy < <mailto:rtoy@google.com> rtoy@google.com>:

I think I'm confused.  The spec says:

 

The number of channels of the output always equals the number of channels of the AudioBuffer assigned to the .buffer attribute, or is one channel of silence if .buffer is NULL.

 

I had interpreted NULL to mean 0. But there's also the Javascript object null.

 

I'm not sure what the spec is saying now, so my comments on your examples maybe be all wrong.

 

Khno> "or is one channel of silence if .buffer is NULL."

I believe that is Javascript object null. That was mentioned a nullable buffer in this code reivew.

If it is not common word expression "nullable", I sorry make you confused.

https://codereview.chromium.org/190953005/

 

In my investigation, source.buffer =0; returns

FF TypeError : "Value being assigned to AudioBufferSourceNode.buffer is not an object"

Chrome TypeError : "Failed to set the 'buffer' property on 'AudioBufferSourceNode': The provided value is not of type 'AudioBuffer'."

 

source.buffer = null; returns

FF : Fine.

Chrome TypeError: "Failed to set the 'buffer' property on 'AudioBufferSourceNode': buffer cannot be null."

 

I would like to fix this one on chrome, setting a nullable buffer.

 

One second thoughts, "there was no "when" which could control accurate timing on setting a buffer at source node that was already started.",

I think we should consider about implementation separately between AudioBufferSourceNode and AudioBuffer.

The "when" in start() or stop()  is specific time for AudioBufferSourceNode, NOT a buffer's start or stop timing.

If JS developers want to make two sound stream with different timing, they should have created two AudioBufferSourceNode as you know.

But, AudioBufferSourceNode should be able to play any AudioBuffers even if there is changing of AudioBuffer on runtime

until finishing to render current AudioBuffers(no loop attributes in sourceNode) or intentional calling stop().

 

In AudioBufferSourceNode perspective, if there is a current buffer which is not reached duration time, it must be running if there is no calling a stop(0 or AudioContext.currentTime).

In addition, if there is stop(AudioContext.currentTime + 20sec), Node must be running until end time.

According to https://github.com/WebAudio/web-audio-api/issues/15, the spec has been changed to allow calling a stop() multiple times differently with start().

"Discussion results: last-called stop() should take effect, i.e. an overwrite of the last stop(). Multiple stop() invokes should not throw, even if the playback has already stopped."

 

It means JS developer can make stopping a AudioBufferSourceNode delay with overwrite stop() with new "when".

 

In spec,

The stop method

Schedules a sound to stop playback at an exact time.

The when parameter describes at what time (in seconds) the sound should stop playing. It is in the same time coordinate system as AudioContext.currentTime. If 0 is passed in for this value or if the value is less thancurrentTime, then the sound will stop playing immediately

 

So, it mentioned AudioContext.currentTime. The "when" is based on AudioContext.currentTime. It means JS developer should call start or stop with "AudioContext.currentTime + delta".

This kind of description supports that "when" is meaningful for AudioBufferSourceNode's life cycle and scheduling, NOT a buffer.

 

I think that FF is working properly below 5 test cases. https://codereview.chromium.org/190953005/, it is giving same behavior to Chrome as FF do except case 5.

 

 

 

 

On Tue, Mar 11, 2014 at 9:42 PM, KeonHo Kim < <mailto:keonho07.kim@samsung.com> keonho07.kim@samsung.com> wrote:


2014-03-12 2:31 GMT+09:00 Raymond Toy < <mailto:rtoy@google.com> rtoy@google.com>:

 

 

On Tue, Mar 11, 2014 at 2:03 AM, KeonHo Kim < <mailto:keonho07.kim@samsung.com> keonho07.kim@samsung.com> wrote:

Dear All.

 

If there is setting a buffer of AudioBufferSourceNode, how it should work?

I think that AudioBufferSourceNode.buffer should be able to set buffer anytime whatever the node is playing or not.

I believe if all cases are able to JS developer, it is fantastic.

 

I think that in all of the cases below, you don't have sample accurate timing. The sound for meaningfulbuffer will start at some uncontrolled time after setting the buffer.  I think in all cases you can get the effect you want by just creating a new AudioBufferSourceNode with the appropriate buffer and calling start and stop appropriately.

 

Chris Rogers certainly intended that AudioBufferSourceNodes to be cheap to create and use. 

 

Khno> I'm getting clear what he intended from discussion with you. I agree with you.

There is no "when" which can control accurate timing.

Don't we need to mention "setBuffer() must be called before calling start()" such like this?.

I think that will be helpful for developer.

 

 

< case1 >
sourcenode.buffer = null;
sourcenode.start(0); // mute
...
sourcenode.buffer = meaningfulbuffer; // sound

 

This case is, I think, currently supported, but I find it's behavior odd because the sound will start at some uncontrolled time.

 

Khno> Actually, chrome returns domexception "ailed to set the 'buffer' property on 'AudioBufferSourceNode': buffer cannot be null" on sourcenode.buffer = null; 

Firefox is not.

< case2 >
sourcenode.buffer = meaningfulbuffer;
sourcenode.start(0); // sound
...
sourcenode.buffer = null; // mute
...
sourcenode.buffer = meaningfulbuffer; // sound

 

I think this is better done by calling sourcenode.stop() instead of setting the buffer to null.  Then create a new new node with meaningfulbuffer.

 

Khno> Yes, it is better approach to keep idea that Chris Rogers intent. 

< case 3 >
sourcenode.start(0);
sourcenode.buffer = meaningfulbuffer; // sound

 

I think this is the same as case 1 

 

Khno> Firefox and Chrome work fine both.

< case 4 >
sourcenode.buffer = meaningfulbuffer;
sourcenode.start(0); // sound

 

This is the normal case. 

 

Khno> Firefox and Chrome work fine both.

< case 5 >
sourcenode.buffer = meaningfulbuffer1;
sourcenode.start(0); // sound buffer1

…

sourcenode.buffer = meaningfulbuffer2; // sound buffer2

 

I don't know how that is supposed to work, especially since start() is only allowed to be called once for each AudioBufferSourceNode.  Even if you allowed more than one call, I still don't know how this is supposed to behave.

 

 

Khno> Firefox and chrome are different.

 

In FF

If there is "sourcenode.buffer = meaningfulbuffer1" then start(0), play meaningfulbuffer1 from begin of buffer.

Then, if there is "sourcenode.buffer = meaningfulbuffer2", sound changed to meaningfulbuffer2 from begin of buffer.

Then, if there is "sourcenode.buffer = meaningfulbuffer1" again, play meaningfulbuffer1 from middle of buffer when "sourcenode.buffer = meaningfulbuffer2" was called.

 

In Chrome

If there is "sourcenode.buffer = meaningfulbuffer1" then start(0), play meaningfulbuffer1 from begin of buffer.

Then, if there is "sourcenode.buffer = meaningfulbuffer2", sound changed to meaningfulbuffer2 from begin of buffer.

Then, if there is "sourcenode.buffer = meaningfulbuffer1" again, play meaningfulbuffer1 from begin of buffer.

 

If a JS developer can not do setting buffer multiple times or not allow setting buffer after calling start() once, FF and Chrome have wrong behavior both.

How do you think this case ? Need to notify "Buffer can not be set more than once" ?

 

 

--

Ray

 

 

 

In spec, nullable buffer is existed which has mono channel silence can help making source muted to AudioBufferSourceNode without stop().

If setting buffer or calling start() is allowed more than once, AudioBufferSourceNode can be reused.

 

Is there any confirmed change about calling start() multiple times?

 

 

However, it also has some issue.

“The spec doesn't say, but the buffer attribute of an AudioBufferSourceNode should probably be only settable once.

If you've started the source node and change the buffer while the previous buffer is playing, you have no control over when the new source starts.”

 <https://github.com/WebAudio/web-audio-api/issues/288> https://github.com/WebAudio/web-audio-api/issues/288

 

Please feel free to give your opinion and correct way for Web Audio API.

 

Br,

Khno

 

 

 

 

From: Raymond Toy [mailto: <mailto:rtoy@google.com> rtoy@google.com] 
Sent: Wednesday, March 12, 2014 2:32 AM
To: KeonHo Kim
Cc:  <mailto:public-audio@w3.org> public-audio@w3.org
Subject: Re: AudioBufferSourceNode.buffer how to work it.

 

 

 

On Tue, Mar 11, 2014 at 2:03 AM, KeonHo Kim < <mailto:keonho07.kim@samsung.com> keonho07.kim@samsung.com> wrote:

Dear All.

 

If there is setting a buffer of AudioBufferSourceNode, how it should work?

I think that AudioBufferSourceNode.buffer should be able to set buffer anytime whatever the node is playing or not.

I believe if all cases are able to JS developer, it is fantastic.

 

I think that in all of the cases below, you don't have sample accurate timing. The sound for meaningfulbuffer will start at some uncontrolled time after setting the buffer.  I think in all cases you can get the effect you want by just creating a new AudioBufferSourceNode with the appropriate buffer and calling start and stop appropriately.

 

Chris Rogers certainly intended that AudioBufferSourceNodes to be cheap to create and use. 

 

< case1 >
sourcenode.buffer = null;
sourcenode.start(0); // mute
...
sourcenode.buffer = meaningfulbuffer; // sound

 

This case is, I think, currently supported, but I find it's behavior odd because the sound will start at some uncontrolled time. 

< case2 >
sourcenode.buffer = meaningfulbuffer;
sourcenode.start(0); // sound
...
sourcenode.buffer = null; // mute
...
sourcenode.buffer = meaningfulbuffer; // sound

 

I think this is better done by calling sourcenode.stop() instead of setting the buffer to null.  Then create a new new node with meaningfulbuffer.

 

< case 3 >
sourcenode.start(0);
sourcenode.buffer = meaningfulbuffer; // sound

 

I think this is the same as case 1 

< case 4 >
sourcenode.buffer = meaningfulbuffer;
sourcenode.start(0); // sound

 

This is the normal case. 

< case 5 >
sourcenode.buffer = meaningfulbuffer1;
sourcenode.start(0); // sound buffer1

…

sourcenode.buffer = meaningfulbuffer2;

sourcenode.start(0); // sound buffer2

 

I don't know how that is supposed to work, especially since start() is only allowed to be called once for each AudioBufferSourceNode.  Even if you allowed more than one call, I still don't know how this is supposed to behave.

 

 

--

Ray

 

 

 

In spec, nullable buffer is existed which has mono channel silence can help making source muted to AudioBufferSourceNode without stop().

If setting buffer or calling start() is allowed more than once, AudioBufferSourceNode can be reused.

 

Is there any confirmed change about calling start() multiple times?

 

 

However, it also has some issue.

“The spec doesn't say, but the buffer attribute of an AudioBufferSourceNode should probably be only settable once.

If you've started the source node and change the buffer while the previous buffer is playing, you have no control over when the new source starts.”

 <https://github.com/WebAudio/web-audio-api/issues/288> https://github.com/WebAudio/web-audio-api/issues/288

 

Please feel free to give your opinion and correct way for Web Audio API.

 

Br,

Khno