Re: Data racing proposals

On Wed, Jul 31, 2013 at 12:46 PM, Chris Wilson <cwilso@google.com> wrote:

> I want to be clear that I still believe (as does Chris) that we should not
> change the current API at this time to address memory sharing across the
> main thread and audio thread with respect to AudioBuffer contents.
> (WaveshaperNode, etc, and worker posting of AudioBuffers are not included
> in this statement.)  Although we respect the design constraint of
> preventing JS thread interference in memory operations, we believe that the
> memory sharing of buffers to the audio thread is not a violation of this
> spirit, and is worth the benefit (and common practice in audio APIs).  If
> the developer community shows that this is a problem in practice,
> particularly as another implementation adds to the corpus of experience, we
> would of course be open to reopening this question at a later date.
>

Keep in mind that the more Web Audio is used, the more difficult this would
be to fix at a later date. Potentially it could become prohibitively
difficult to fix.

However, independently I'd like to comment on the various proposals.  In
> the event the group believes we should be preventing memory sharing now, I
> would like to suggest an alternate proposal (though I want to be clear that
> my strong preference is still to not change the API without proof this is a
> problem in practice).
>

Thanks for doing this. Your intent is clear.


> I believe this proposal has a fair amount in common with ROC's proposal,
> and some with Jer's proposal - however, I've been exclusively looking at
> this from the point of view of the web developer and how they would use the
> API.  The problem I see with ROC's proposal (and thanks to ROC and Ehsan
> for patiently elaborating on that) is that it is not clear to the developer
> when a copy operation is performed,
>

This I grant you...


> and they need to do something different with the ArrayBuffer returned from
> getChannelData (i.e. re-assign it to another AudioBuffer);
>

That's not correct. With my proposal, when getChannelData is called while
or after an AudioBuffer is in use, the app can modify the Float32Array and
then reuse the same AudioBuffer with those modifications taken into account.

This proposal is:
> AudioBuffer's contents continue to be exposed and used in the same way
> prior to playback.  Implementations continue to decode or create
> AudioBuffers in the same way, and continue to read (and write) contents
> with the same getChannelData method, which allows access to the memory
> buffers themselves to the main UI thread.
>
> However: once playback of the AudioBuffer has been scheduled (i.e.,
> "start" is called on any BufferSourceNode that has its .buffer assigned to
> this AudioBuffer), the AudioBuffer's internal Float32Arrays are neutered
> and no longer available for direct access [a][b].  Subsequent attempts to
> access the ArrayBuffer contents will throw; subsequent calls to
> getChannelData will also throw.  [c]  For the scenarios where this data
> needs to be accessed after playback has been scheduled, however (e.g.
> waveform display), a new method should be added on AudioBuffer:
>
>     partial interface AudioBuffer {
>         Promise clone( optional begin, optional end );   // Fulfills with
> a value of type AudioBuffer, with the same channel layout as the original
> AudioBuffer
>     }
>
> which will create a copy of [a slice of] the data, as a separate
> AudioBuffer (which, of course, is not neutered to begin with).  The slicing
> capability is critical for minimizing in-place memory of small sections of
> audio requested while the audio is being played - such as live waveform
> display.
>

This is a reasonable option IMHO. Its main advantage over my proposal is
that it simplifies the performance model by making the implicit copies of
my proposal explicit. OTOH it gives up some compatibility and makes the API
a little harder to use.

I think we could improve this approach a bit. We don't really need a
Promise here since AudioBuffer contents are expected to be in memory. It
would also be good to be able to reuse an existing Float32Array instead of
having to allocate a new one. So I'd suggest something like "void
copyTo(Float32Array dest, long offset, long length)" instead.

Additionally: in the case when the AudioBuffer's contents are transferred
> to another thread, the AudioBuffer is again neutered and no longer
> available for direct access.  In this case, once neutered in this way, the
> AudioBuffer cannot be played (but, of course, you could request a clone to
> be made).
>

I'm not sure what you're getting at here. What exactly do you mean by "when
the AudioBuffer's contents are transferred to another thread"?

In both of these cases, note the use of an asynchronous Promise for the
> cloning operation; in some implementations, the memory may have moved
> across thread boundaries, and need to be copied back across those
> boundaries.
>

I think we can assume memory copies between threads can be performed
synchronously.

[a] Optional modifier - once playback has ended, you could un-neuter the
> ArrayBuffer contents.  I'll resist the temptation to come up with a
> creative term for this.  This seems challenging to define (as poorly as
> neutering appears to be defined, there's no precedent I could find for
> un-neutering), however, so not part of the main proposal
>

In your proposal, that would be an observable behavior change. In my
proposal, a UA can do the same thing, but it's not observable --- the
getChannelData() just becomes nearly-free instead of having to copy, by
creating a new ArrayBuffer with the data contents stolen back from the
audio thread. I think this is a slight advantage for my proposal.

[b] Optional modifier - make the neutering occur when the AudioBuffer is
> assigned to any BufferSourceNode's .buffer, as in ROC's original proposal.
>  I prefer to reserve this as long as possible, but I do not feel this is a
> critical point.
>

I agree it's not critical. I would be fine with altering my proposal to
match yours here.

Rob
-- 
Jtehsauts  tshaei dS,o n" Wohfy  Mdaon  yhoaus  eanuttehrotraiitny  eovni
le atrhtohu gthot sf oirng iyvoeu rs ihnesa.r"t sS?o  Whhei csha iids  teoa
stiheer :p atroa lsyazye,d  'mYaonu,r  "sGients  uapr,e  tfaokreg iyvoeunr,
'm aotr  atnod  sgaoy ,h o'mGee.t"  uTph eann dt hwea lmka'n?  gBoutt  uIp
waanndt  wyeonut  thoo mken.o w  *
*