Re: AudioBufferSourceNode.buffer how to work it.

I'm not allergic to defining this some other way; however, I do think we
need to very carefully define it.  Primarily, I'm interested in what
happens when you call start(0) with no buffer, and at some later point set
the buffer.


On Mon, Mar 17, 2014 at 9:18 PM, Karl Tomlinson <
karlt+public-audio@karlt.net> wrote:

> The spec currently doesn't require any particular ordering of
> setting the buffer and calling start(), and so introducing a
> new requirement would affect backward compatibility.
>
> This includes the issue identified in
> http://lists.w3.org/Archives/Public/public-audio/2014JanMar/0063.html
>
> Requiring all changes (except stop()) to the AudioBufferSourceNode
> to be made before calling start() would certainly make things
> simpler for implementations, but IMO we should have a stronger
> reason before making breaking changes.
>
> Chris Wilson writes:
>
> > I actually think start() should throw if the buffer is null.
> >
> >
> > On Mon, Mar 17, 2014 at 2:50 PM, Raymond Toy <rtoy@google.com> wrote:
> >
> >> I also agree. But note that case 1:
> >>
> >> < case1 >
> >> sourcenode.buffer = null;
> >> sourcenode.start(0); // mute
> >> ...
> >> sourcenode.buffer = meaningfulbuffer; // sound
> >>
> >> I personally feel this should not be allowed either.
> >>
> >>
> >> On Mon, Mar 17, 2014 at 12:02 PM, Chris Wilson <cwilso@google.com>
> wrote:
> >>
> >>> That would be my suggestion, yes.  Others should weigh in, though.
> >>>
> >>>
> >>> On Sun, Mar 16, 2014 at 11:43 PM, KeonHo Kim <keonho07.kim@samsung.com
> >wrote:
> >>>
> >>>> @Chris Wilson.
> >>>>
> >>>>
> >>>>
> >>>> Thanks for clarification.
> >>>>
> >>>> Can I expect one of future draft will be tried to included more
> explicit
> >>>> description about this issue.?
> >>>>
> >>>> According to your opinion below 4 cases should throw exception. My
> >>>> understanding is correct ? J
> >>>>
> >>>>
> >>>>
> >>>> < case1 >
> >>>> sourcenode.buffer = null;
> >>>> sourcenode.start(0);
> >>>> ...
> >>>> sourcenode.buffer = meaningfulbuffer; // Exception “It has been tried
> to
> >>>> set buffer after AudioBufferSourceNode is already started.”
> >>>>
> >>>>
> >>>>
> >>>> < case2 >
> >>>> sourcenode.buffer = meaningfulbuffer;
> >>>> sourcenode.start(0);
> >>>> ...
> >>>> sourcenode.buffer = null; // Exception “It has been tried to set
> buffer
> >>>> after AudioBufferSourceNode is already started.”
> >>>> ...
> >>>> sourcenode.buffer = meaningfulbuffer; // Exception “It has been tried
> to
> >>>> set buffer after AudioBufferSourceNode is already started.”
> >>>>
> >>>>
> >>>>
> >>>> < case 3 >
> >>>> sourcenode.start(0);
> >>>> sourcenode.buffer = meaningfulbuffer; // Exception “It has been tried
> to
> >>>> set buffer after AudioBufferSourceNode is already started.”
> >>>>
> >>>>
> >>>>
> >>>> < case 5 >
> >>>> sourcenode.buffer = meaningfulbuffer1;
> >>>> sourcenode.start(0); // sound buffer1
> >>>>
> >>>> sourcenode.buffer = meaningfulbuffer2; // Exception “It has been tried
> >>>> to set buffer after AudioBufferSourceNode is already started.”
> >>>>
> >>>>
> >>>>
> >>>> Br,
> >>>>
> >>>> KeonHo
> >>>>
> >>>> *From:* Chris Wilson [mailto:cwilso@google.com]
> >>>> *Sent:* Saturday, March 15, 2014 6:48 AM
> >>>> *To:* KeonHo Kim
> >>>> *Cc:* Raymond Toy; public-audio@w3.org
> >>>>
> >>>> *Subject:* Re: AudioBufferSourceNode.buffer how to work it.
> >>>>
> >>>>
> >>>>
> >>>> "null" refers to the Javascript null object; perhaps we need to be
> more
> >>>> explicit about that.
> >>>>
> >>>>
> >>>>
> >>>> I have no idea why any implementation would re-start the buffer once
> >>>> it's already playing; personally, I think we should explicitly ignore
> >>>> setting the buffer once it has been set.
> >>>>
> >>>>
> >>>>
> >>>> On Wed, Mar 12, 2014 at 6:51 PM, KeonHo Kim <keonho07.kim@samsung.com
> >
> >>>> wrote:
> >>>>
> >>>>
> >>>>
> >>>> 2014-03-13 3:01 GMT+09:00 Raymond Toy <rtoy@google.com>:
> >>>>
> >>>> I think I'm confused.  The spec says:
> >>>>
> >>>>
> >>>>
> >>>> The number of channels of the output always equals the number of
> >>>> channels of the AudioBuffer assigned to the .buffer attribute, or is
> one
> >>>> channel of silence if .buffer is NULL.
> >>>>
> >>>>
> >>>>
> >>>> I had interpreted NULL to mean 0. But there's also the Javascript
> object
> >>>> null.
> >>>>
> >>>>
> >>>>
> >>>> I'm not sure what the spec is saying now, so my comments on your
> >>>> examples maybe be all wrong.
> >>>>
> >>>>
> >>>>
> >>>> Khno> "or is one channel of silence if .buffer is NULL."
> >>>>
> >>>> I believe that is Javascript object null. That was mentioned a
> nullable
> >>>> buffer in this code reivew.
> >>>>
> >>>> If it is not common word expression "nullable", I sorry make you
> >>>> confused.
> >>>>
> >>>> https://codereview.chromium.org/190953005/
> >>>>
> >>>>
> >>>>
> >>>> In my investigation, source.buffer =0; returns
> >>>>
> >>>> FF TypeError : "Value being assigned to AudioBufferSourceNode.buffer
> is
> >>>> not an object"
> >>>>
> >>>> Chrome TypeError : "Failed to set the 'buffer' property on
> >>>> 'AudioBufferSourceNode': The provided value is not of type
> 'AudioBuffer'."
> >>>>
> >>>>
> >>>>
> >>>> source.buffer = null; returns
> >>>>
> >>>> FF : Fine.
> >>>>
> >>>> Chrome TypeError: "Failed to set the 'buffer' property on
> >>>> 'AudioBufferSourceNode': buffer cannot be null."
> >>>>
> >>>>
> >>>>
> >>>> I would like to fix this one on chrome, setting a nullable buffer.
> >>>>
> >>>>
> >>>>
> >>>> One second thoughts, "there was no "when" which could control accurate
> >>>> timing on setting a buffer at source node that was already started.",
> >>>>
> >>>> I think we should consider about implementation separately between
> >>>> AudioBufferSourceNode and AudioBuffer.
> >>>>
> >>>> The "when" in start() or stop()  is specific time
> >>>> for AudioBufferSourceNode, NOT a buffer's start or stop timing.
> >>>>
> >>>> If JS developers want to make two sound stream with different timing,
> >>>> they should have created two AudioBufferSourceNode as you know.
> >>>>
> >>>> But, AudioBufferSourceNode should be able to play any AudioBuffers
> even
> >>>> if there is changing of AudioBuffer on runtime
> >>>>
> >>>> until finishing to render current AudioBuffers(no loop attributes in
> >>>> sourceNode) or intentional calling stop().
> >>>>
> >>>>
> >>>>
> >>>> In AudioBufferSourceNode perspective, if there is a current buffer
> which
> >>>> is not reached duration time, it must be running if there is no
> calling a
> >>>> stop(0 or AudioContext.currentTime).
> >>>>
> >>>> In addition, if there is stop(AudioContext.currentTime + 20sec), Node
> >>>> must be running until end time.
> >>>>
> >>>> According to https://github.com/WebAudio/web-audio-api/issues/15, the
> >>>> spec has been changed to allow calling a stop() multiple times
> differently
> >>>> with start().
> >>>>
> >>>> "Discussion results: last-called stop() should take effect, i.e. an
> >>>> overwrite of the last stop(). Multiple stop() invokes should not
> throw,
> >>>> even if the playback has already stopped."
> >>>>
> >>>>
> >>>>
> >>>> It means JS developer can make stopping a AudioBufferSourceNode delay
> >>>> with overwrite stop() with new "when".
> >>>>
> >>>>
> >>>>
> >>>> In spec,
> >>>>
> >>>> The stop method
> >>>>
> >>>> Schedules a sound to stop playback at an exact time.
> >>>>
> >>>> The *when* parameter describes at what time (in seconds) the sound
> >>>> should stop playing. It is in the same time coordinate system as
> >>>> AudioContext.currentTime. If 0 is passed in for this value or if the
> value
> >>>> is less than*currentTime*, then the sound will stop playing
> immediately
> >>>>
> >>>>
> >>>>
> >>>> So, it mentioned AudioContext.currentTime. The "when" is based
> >>>> on AudioContext.currentTime. It means JS developer should call start
> or
> >>>> stop with "AudioContext.currentTime + delta".
> >>>>
> >>>> This kind of description supports that "when" is meaningful for
> >>>> AudioBufferSourceNode's life cycle and scheduling, NOT a buffer.
> >>>>
> >>>>
> >>>>
> >>>> I think that FF is working properly below 5 test cases.
> >>>> https://codereview.chromium.org/190953005/, it is giving same
> behavior
> >>>> to Chrome as FF do except case 5.
> >>>>
> >>>>
> >>>>
> >>>>
> >>>>
> >>>>
> >>>>
> >>>>
> >>>>
> >>>> On Tue, Mar 11, 2014 at 9:42 PM, KeonHo Kim <keonho07.kim@samsung.com
> >>>> > wrote:
> >>>>
> >>>>
> >>>> 2014-03-12 2:31 GMT+09:00 Raymond Toy <rtoy@google.com>:
> >>>>
> >>>>
> >>>>
> >>>>
> >>>>
> >>>> On Tue, Mar 11, 2014 at 2:03 AM, KeonHo Kim <keonho07.kim@samsung.com
> >>>> > wrote:
> >>>>
> >>>> Dear All.
> >>>>
> >>>>
> >>>>
> >>>> If there is setting a buffer of AudioBufferSourceNode, how it should
> >>>> work?
> >>>>
> >>>> I think that AudioBufferSourceNode.buffer should be able to set buffer
> >>>> anytime whatever the node is playing or not.
> >>>>
> >>>> I believe if all cases are able to JS developer, it is fantastic.
> >>>>
> >>>>
> >>>>
> >>>> I think that in all of the cases below, you don't have sample accurate
> >>>> timing. The sound for meaningfulbuffer will start at some
> uncontrolled time
> >>>> after setting the buffer.  I think in all cases you can get the
> effect you
> >>>> want by just creating a new AudioBufferSourceNode with the appropriate
> >>>> buffer and calling start and stop appropriately.
> >>>>
> >>>>
> >>>>
> >>>> Chris Rogers certainly intended that AudioBufferSourceNodes to be
> cheap
> >>>> to create and use.
> >>>>
> >>>>
> >>>>
> >>>> Khno> I'm getting clear what he intended from discussion with you. I
> >>>> agree with you.
> >>>>
> >>>> There is no "when" which can control accurate timing.
> >>>>
> >>>> Don't we need to mention "setBuffer() must be called before calling
> >>>> start()" such like this?.
> >>>>
> >>>> I think that will be helpful for developer.
> >>>>
> >>>>
> >>>>
> >>>>
> >>>>
> >>>> < case1 >
> >>>> sourcenode.buffer = null;
> >>>> sourcenode.start(0); // mute
> >>>> ...
> >>>> sourcenode.buffer = meaningfulbuffer; // sound
> >>>>
> >>>>
> >>>>
> >>>> This case is, I think, currently supported, but I find it's behavior
> odd
> >>>> because the sound will start at some uncontrolled time.
> >>>>
> >>>>
> >>>>
> >>>> Khno> Actually, chrome returns domexception "ailed to set the 'buffer'
> >>>> property on 'AudioBufferSourceNode': buffer cannot be null" on
> >>>> sourcenode.buffer
> >>>> = null;
> >>>>
> >>>> Firefox is not.
> >>>>
> >>>> < case2 >
> >>>> sourcenode.buffer = meaningfulbuffer;
> >>>> sourcenode.start(0); // sound
> >>>> ...
> >>>> sourcenode.buffer = null; // mute
> >>>> ...
> >>>> sourcenode.buffer = meaningfulbuffer; // sound
> >>>>
> >>>>
> >>>>
> >>>> I think this is better done by calling sourcenode.stop() instead of
> >>>> setting the buffer to null.  Then create a new new node with
> >>>> meaningfulbuffer.
> >>>>
> >>>>
> >>>>
> >>>> Khno> Yes, it is better approach to keep idea that Chris Rogers
> intent.
> >>>>
> >>>> < case 3 >
> >>>> sourcenode.start(0);
> >>>> sourcenode.buffer = meaningfulbuffer; // sound
> >>>>
> >>>>
> >>>>
> >>>> I think this is the same as case 1
> >>>>
> >>>>
> >>>>
> >>>> Khno> Firefox and Chrome work fine both.
> >>>>
> >>>> < case 4 >
> >>>> sourcenode.buffer = meaningfulbuffer;
> >>>> sourcenode.start(0); // sound
> >>>>
> >>>>
> >>>>
> >>>> This is the normal case.
> >>>>
> >>>>
> >>>>
> >>>> Khno> Firefox and Chrome work fine both.
> >>>>
> >>>> < case 5 >
> >>>> sourcenode.buffer = meaningfulbuffer1;
> >>>> sourcenode.start(0); // sound buffer1
> >>>>
> >>>> …
> >>>>
> >>>> sourcenode.buffer = meaningfulbuffer2; // sound buffer2
> >>>>
> >>>>
> >>>>
> >>>> I don't know how that is supposed to work, especially since start() is
> >>>> only allowed to be called once for each AudioBufferSourceNode.  Even
> if you
> >>>> allowed more than one call, I still don't know how this is supposed to
> >>>> behave.
> >>>>
> >>>>
> >>>>
> >>>>
> >>>>
> >>>> Khno> Firefox and chrome are different.
> >>>>
> >>>>
> >>>>
> >>>> In FF
> >>>>
> >>>> If there is "sourcenode.buffer = meaningfulbuffer1" then start(0),
> play
> >>>> meaningfulbuffer1
> >>>> from begin of buffer.
> >>>>
> >>>> Then, if there is "sourcenode.buffer = meaningfulbuffer2", sound
> >>>> changed to meaningfulbuffer2 from begin of buffer.
> >>>>
> >>>> Then, if there is "sourcenode.buffer = meaningfulbuffer1" again, play
> >>>> meaningfulbuffer1
> >>>> from middle of buffer when "sourcenode.buffer = meaningfulbuffer2" was
> >>>> called.
> >>>>
> >>>>
> >>>>
> >>>> In Chrome
> >>>>
> >>>> If there is "sourcenode.buffer = meaningfulbuffer1" then start(0),
> play
> >>>> meaningfulbuffer1
> >>>> from begin of buffer.
> >>>>
> >>>> Then, if there is "sourcenode.buffer = meaningfulbuffer2", sound
> >>>> changed to meaningfulbuffer2 from begin of buffer.
> >>>>
> >>>> Then, if there is "sourcenode.buffer = meaningfulbuffer1" again, play
> >>>> meaningfulbuffer1
> >>>> from begin of buffer.
> >>>>
> >>>>
> >>>>
> >>>> If a JS developer can not do setting buffer multiple times or not
> allow
> >>>> setting buffer after calling start() once, FF and Chrome have wrong
> >>>> behavior both.
> >>>>
> >>>> How do you think this case ? Need to notify "Buffer can not be set
> more
> >>>> than once" ?
> >>>>
> >>>>
> >>>>
> >>>>
> >>>>
> >>>> --
> >>>>
> >>>> Ray
> >>>>
> >>>>
> >>>>
> >>>>
> >>>>
> >>>>
> >>>>
> >>>> In spec, nullable buffer is existed which has mono channel silence can
> >>>> help making source muted to AudioBufferSourceNode without stop().
> >>>>
> >>>> If setting buffer or calling start() is allowed more than once,
> >>>> AudioBufferSourceNode can be reused.
> >>>>
> >>>>
> >>>>
> >>>> Is there any confirmed change about calling start() multiple times?
> >>>>
> >>>>
> >>>>
> >>>>
> >>>>
> >>>> However, it also has some issue.
> >>>>
> >>>> “The spec doesn't say, but the buffer attribute of an
> >>>> AudioBufferSourceNode should probably be only settable once.
> >>>>
> >>>> If you've started the source node and change the buffer while the
> >>>> previous buffer is playing, you have no control over when the new
> source
> >>>> starts.”
> >>>>
> >>>> https://github.com/WebAudio/web-audio-api/issues/288
> >>>>
> >>>>
> >>>>
> >>>> Please feel free to give your opinion and correct way for Web Audio
> API.
> >>>>
> >>>>
> >>>>
> >>>> Br,
> >>>>
> >>>> Khno
> >>>>
> >>>>
> >>>>
> >>>>
> >>>>
> >>>>
> >>>>
> >>>>
> >>>>
> >>>> *From:* Raymond Toy [mailto:rtoy@google.com]
> >>>> *Sent:* Wednesday, March 12, 2014 2:32 AM
> >>>> *To:* KeonHo Kim
> >>>> *Cc:* public-audio@w3.org
> >>>> *Subject:* Re: AudioBufferSourceNode.buffer how to work it.
> >>>>
> >>>>
> >>>>
> >>>>
> >>>>
> >>>>
> >>>>
> >>>> On Tue, Mar 11, 2014 at 2:03 AM, KeonHo Kim <keonho07.kim@samsung.com
> >
> >>>> wrote:
> >>>>
> >>>> Dear All.
> >>>>
> >>>>
> >>>>
> >>>> If there is setting a buffer of AudioBufferSourceNode, how it should
> >>>> work?
> >>>>
> >>>> I think that AudioBufferSourceNode.buffer should be able to set buffer
> >>>> anytime whatever the node is playing or not.
> >>>>
> >>>> I believe if all cases are able to JS developer, it is fantastic.
> >>>>
> >>>>
> >>>>
> >>>> I think that in all of the cases below, you don't have sample accurate
> >>>> timing. The sound for meaningfulbuffer will start at some
> uncontrolled time
> >>>> after setting the buffer.  I think in all cases you can get the
> effect you
> >>>> want by just creating a new AudioBufferSourceNode with the appropriate
> >>>> buffer and calling start and stop appropriately.
> >>>>
> >>>>
> >>>>
> >>>> Chris Rogers certainly intended that AudioBufferSourceNodes to be
> cheap
> >>>> to create and use.
> >>>>
> >>>>
> >>>>
> >>>> < case1 >
> >>>> sourcenode.buffer = null;
> >>>> sourcenode.start(0); // mute
> >>>> ...
> >>>> sourcenode.buffer = meaningfulbuffer; // sound
> >>>>
> >>>>
> >>>>
> >>>> This case is, I think, currently supported, but I find it's behavior
> odd
> >>>> because the sound will start at some uncontrolled time.
> >>>>
> >>>> < case2 >
> >>>> sourcenode.buffer = meaningfulbuffer;
> >>>> sourcenode.start(0); // sound
> >>>> ...
> >>>> sourcenode.buffer = null; // mute
> >>>> ...
> >>>> sourcenode.buffer = meaningfulbuffer; // sound
> >>>>
> >>>>
> >>>>
> >>>> I think this is better done by calling sourcenode.stop() instead of
> >>>> setting the buffer to null.  Then create a new new node with
> >>>> meaningfulbuffer.
> >>>>
> >>>>
> >>>>
> >>>> < case 3 >
> >>>> sourcenode.start(0);
> >>>> sourcenode.buffer = meaningfulbuffer; // sound
> >>>>
> >>>>
> >>>>
> >>>> I think this is the same as case 1
> >>>>
> >>>> < case 4 >
> >>>> sourcenode.buffer = meaningfulbuffer;
> >>>> sourcenode.start(0); // sound
> >>>>
> >>>>
> >>>>
> >>>> This is the normal case.
> >>>>
> >>>> < case 5 >
> >>>> sourcenode.buffer = meaningfulbuffer1;
> >>>> sourcenode.start(0); // sound buffer1
> >>>>
> >>>> …
> >>>>
> >>>> sourcenode.buffer = meaningfulbuffer2;
> >>>>
> >>>> sourcenode.start(0); // sound buffer2
> >>>>
> >>>>
> >>>>
> >>>> I don't know how that is supposed to work, especially since start() is
> >>>> only allowed to be called once for each AudioBufferSourceNode.  Even
> if you
> >>>> allowed more than one call, I still don't know how this is supposed to
> >>>> behave.
> >>>>
> >>>>
> >>>>
> >>>>
> >>>>
> >>>> --
> >>>>
> >>>> Ray
> >>>>
> >>>>
> >>>>
> >>>>
> >>>>
> >>>>
> >>>>
> >>>> In spec, nullable buffer is existed which has mono channel silence can
> >>>> help making source muted to AudioBufferSourceNode without stop().
> >>>>
> >>>> If setting buffer or calling start() is allowed more than once,
> >>>> AudioBufferSourceNode can be reused.
> >>>>
> >>>>
> >>>>
> >>>> Is there any confirmed change about calling start() multiple times?
> >>>>
> >>>>
> >>>>
> >>>>
> >>>>
> >>>> However, it also has some issue.
> >>>>
> >>>> “The spec doesn't say, but the buffer attribute of an
> >>>> AudioBufferSourceNode should probably be only settable once.
> >>>>
> >>>> If you've started the source node and change the buffer while the
> >>>> previous buffer is playing, you have no control over when the new
> source
> >>>> starts.”
> >>>>
> >>>> https://github.com/WebAudio/web-audio-api/issues/288
> >>>>
> >>>>
> >>>>
> >>>> Please feel free to give your opinion and correct way for Web Audio
> API.
> >>>>
> >>>>
> >>>>
> >>>> Br,
> >>>>
> >>>> Khno
> >>>>
> >>>>
> >>>>
> >>>>
> >>>>
> >>>>
> >>>>
> >>>>
> >>>>
> >>>>
> >>>>
> >>>
> >>>
> >>
>