Re: TAG feedback on Web Audio

2013/8/6 Chris Wilson <cwilso@google.com>

> I still (obviously) disagree that a model that relies on passing copies
> around will have the same memory/speed/latency performance.
>

Sorry, maybe I wasn't clear enough... I meant as an example that
*internally* you could pass *references to read-only buffers* around (which
has about zero overhead), in order to highlight the fact that sharing of
mutable data between threads is *not* a requirement for implementing an
audio API with good memory/speed/latency performance.


 It's possible the actual copying can be minimized, at a cost in usability
> to the developer, but even with that I don't think the cost will ever be
> truly zero; so this is a tradeoff.
>

Of course it's a trade off, just as using 32-bit floats instead of 64-bit
floats or 16-bit fixed points was a tradeoff. In the end we aim to hit the
"sweet spot".

Regarding "cost in usability", that's a fairly subjective statement. I
personally find the current API quite non-intuitive to work with, much
because it uses lots of non-JS constructs (the getChannelData() method, for
instance, works in a way that I've never seen on the Web before).



>  It may be that the group chooses that tradeoff - but traditional audio
> APIs have not.
>

I guess that depends on what you mean with "traditional audio APIs".
OpenAL, for instance, uses explicit copying.


/Marcus



>  Of course, most of them don't have the limited execution scope of JS,
> either.  (Note that I should have said "glitch-free, reasonably-low-latency
> audio with good performance and API usability." :)
>
> Interestingly enough, if we'd not allowed JS in nodes (rather used a more
> restricted audio processing language), I'm not sure we'd have this problem
> to this scale.
>
>
> On Tue, Aug 6, 2013 at 2:43 AM, Marcus Geelnard <mage@opera.com> wrote:
>
>>  2013-08-06 04:20, Chris Wilson skrev:
>>
>> See, I read two different things from this.  Marcus, I heard you say the
>> comment about not being describable in JS was in specific reference to
>> threading and memory sharing not being part of the current model; from
>> Alex, I heard "open your mind to what JS *could* be, and then describe
>> everything in one model" - in part, the nodes' behaviors themselves should
>> be described in JS, albeit of course not necessarily run in JS.
>>
>>  The latter (as I'd previously expressed to Alex) is fine with me; I'd
>> be fine describing how delay nodes work in JS.  However, high-quality
>> glitch-free audio requires a number of things that aren't available to JS
>> today - notably, a separate thread with restricted execution capabilities
>> (i.e. not arbitrary JavaScript) and no garbage collection that can have
>> elevated priority, and access to the physical hardware, of course.  It's
>> not at all impossible to imagine how to describe Web Audio if you have
>> those capabilities; and the memory sharing we're discussing under the
>> rubric of race conditions is really whether that somewhat different
>> execution environment supports those or not.
>>
>>
>> I have always been a strong supporter of being able to describe and
>> implement as much as possible of the API in JS, and I agree that there are
>> certainly some important aspects missing in the JS execution environment
>> that make a practical implementation in JS impossible (including access to
>> audio HW and low-level control over thread execution etc). You should,
>> however, still be able to implement the *behavior* of the API in JS (as
>> an exercise - imagine implementing an OfflineAudioContext in JS - I don't
>> see why it shouldn't be possible).
>>
>> I might have misinterpreted Alex's point (I thought he was referring to
>> the shared data in the API interfaces, but of course there may be more
>> things to it). However, from the perspective of making the Web Audio API
>> describable in JS terms, I still think that the issue of exposing shared
>> data interfaces to JS is a key deficiency in the current design, mostly
>> because:
>>
>> * It makes it impossible to express operations such as
>> AudioBuffer.getChannelData() in JS terms.
>> * Since the interfaces are exposed to JS, the *only* way to make them
>> JS-friendly would be to change/extend the JS memory model (which is a much
>> bigger task than to change the Web Audio API model).
>> * It's not in any way necessary for achieving the goal "glitch-free,
>> reasonably-low-latency audio".
>>
>> In fact, I believe it should be fully possible to implement the Web Audio
>> API without using shared *mutable* data internally (provided that we
>> drop the shared data model from the interfaces). Correct me if I'm wrong,
>> but I'm pretty sure you could solve most things by passing read-only
>> references to data buffers between threads (which would be semantically
>> equivalent to passing copies around) and still have the same
>> memory/speed/latency performance. That would make it a moot point whether
>> or not shared data must be part of a potential fictional execution
>> environment or not.
>>
>> /Marcus
>>
>>
>>
>>
>>
>> On Mon, Aug 5, 2013 at 1:49 PM, Alex Russell <slightlyoff@google.com>wrote:
>>
>>>  On Sun, Aug 4, 2013 at 6:46 PM, Chris Wilson <cwilso@google.com> wrote:
>>>
>>>> (Sorry, on vacation, and beach > Web Audio discussions. :)
>>>>
>>>>  Alex, as we discussed a few weeks ago - glitch-free,
>>>> reasonably-low-latency audio is something that I just don't believe JS -
>>>> *as it is today - *can do.
>>>>
>>>
>>>  In the TAG feedback document you might have detected two lines of
>>> attack on this argument:
>>>
>>>
>>>    1. It's insufficient to say "JavaScript can't do this". The entire
>>>    goal of some of the suggestions in the document related to the execution
>>>    model are all about creating space in the model to change the constraints
>>>    under which JS (or something else) runs such that JS* *(or ASMJS or
>>>    NaCL in a worker, etc.) *could* do what's needed without breaking
>>>    the invariants of either the platform or the conceptual semantic model that
>>>    Web Audio presents. So that's simply an argument against an assertion
>>>    *that hasn't been presented*. It fails on that ground alone,
>>>    however...
>>>    2. Nothing in my argument against breaking the platform invariants
>>>    requires that you actually RUN the built-in node types in JS. To some great
>>>    degree, I'm advocating for JS-as-spec-language when I discuss de-sugaring.
>>>    Not an implementation strategy (except perhaps in naive cases for
>>>    newly-bootstrapping impls).
>>>
>>> The net is that discussing main-thread-JS-as-we-know-it-today and not
>>> the framework for execution/specification is arguing against an
>>> implementation-level straw-man.
>>>
>>>  Regards
>>>
>>>    On Thu, Aug 1, 2013 at 5:42 PM, Alex Russell <slightlyoff@google.com>wrote:
>>>>
>>>>> Let me be clearer, then: the issue is that it introduces effects to JS
>>>>> that can't be described *in terms of JS*. It violates the
>>>>> run-to-completion model of the language without appealing to the turn-based
>>>>> concurrency model we use everywhere else in the platform.
>>>>>
>>>>>
>>>>> On Wed, Jul 31, 2013 at 10:43 AM, Chris Wilson <cwilso@google.com>wrote:
>>>>>
>>>>>> In addition, I'd ask that you be more explicit than calling this
>>>>>> problem "data races", because there's clearly some explicit effect you're
>>>>>> trying to prevent.  Any asynchronously-programmable or event-driven system
>>>>>> can enable developers to introduce race conditions.
>>>>>>
>>>>>>
>>>>>> On Mon, Jul 29, 2013 at 10:09 PM, Noah Mendelsohn <
>>>>>> nrm@arcanedomain.com> wrote:
>>>>>>
>>>>>>>
>>>>>>>
>>>>>>> On 7/29/2013 7:05 PM, Anne van Kesteren wrote:
>>>>>>>
>>>>>>>> On Sat, Jul 27, 2013 at 9:22 AM, Noah Mendelsohn<
>>>>>>>> nrm@arcanedomain.com>  wrote:
>>>>>>>>
>>>>>>>>> >Again, I have no informed opinions on the specific merits, just
>>>>>>>>> suggesting a
>>>>>>>>> >useful role the TAG might play to clarify for the many members of
>>>>>>>>> the
>>>>>>>>> >community who are less expert on this than you are. Thank you.
>>>>>>>>>
>>>>>>>>
>>>>>>>  I'm not sure we call out data races anywhere, it's something we
>>>>>>>> just don't do.
>>>>>>>>
>>>>>>>
>>>>>>>  Well, my recollection may be faulty, but I think that one of the
>>>>>>> reasons the TAG took the trouble to formalize things like the architecture
>>>>>>> document was the belief that it's easier to ask skeptics to stick to rules
>>>>>>> that have been written down, and especially those that have garnered formal
>>>>>>> consensus through something like the Recommendation track.
>>>>>>>
>>>>>>> Whether it's worth taking a guideline on data races all the way to
>>>>>>> Rec I'm not sure, but it seems that it would be worth setting it down
>>>>>>> formally, perhaps in a TAG Finding/blog post/Recommendation or whatever
>>>>>>> will get the right level of discussion, consensus building, and eventually
>>>>>>> attention.
>>>>>>>
>>>>>>> Certainly, of the many things that have come up recently relating to
>>>>>>> APIs, this one seems deeply architectural and very much within the TAG's
>>>>>>> remit.
>>>>>>>
>>>>>>> Noah
>>>>>>>
>>>>>>>
>>>>>>
>>>>>
>>>>
>>>
>>
>>
>> --
>> Marcus Geelnard
>> Technical Lead, Mobile Infrastructure
>> Opera Software
>>
>>
>