Re: TAG feedback on Web Audio

2013-08-06 04:20, Chris Wilson skrev:
> See, I read two different things from this.  Marcus, I heard you say 
> the comment about not being describable in JS was in specific 
> reference to threading and memory sharing not being part of the 
> current model; from Alex, I heard "open your mind to what JS *could* 
> be, and then describe everything in one model" - in part, the nodes' 
> behaviors themselves should be described in JS, albeit of course not 
> necessarily run in JS.
>
> The latter (as I'd previously expressed to Alex) is fine with me; I'd 
> be fine describing how delay nodes work in JS.  However, high-quality 
> glitch-free audio requires a number of things that aren't available to 
> JS today - notably, a separate thread with restricted execution 
> capabilities (i.e. not arbitrary JavaScript) and no garbage collection 
> that can have elevated priority, and access to the physical hardware, 
> of course.  It's not at all impossible to imagine how to describe Web 
> Audio if you have those capabilities; and the memory sharing we're 
> discussing under the rubric of race conditions is really whether that 
> somewhat different execution environment supports those or not.

I have always been a strong supporter of being able to describe and 
implement as much as possible of the API in JS, and I agree that there 
are certainly some important aspects missing in the JS execution 
environment that make a practical implementation in JS impossible 
(including access to audio HW and low-level control over thread 
execution etc). You should, however, still be able to implement the 
/behavior/ of the API in JS (as an exercise - imagine implementing an 
OfflineAudioContext in JS - I don't see why it shouldn't be possible).

I might have misinterpreted Alex's point (I thought he was referring to 
the shared data in the API interfaces, but of course there may be more 
things to it). However, from the perspective of making the Web Audio API 
describable in JS terms, I still think that the issue of exposing shared 
data interfaces to JS is a key deficiency in the current design, mostly 
because:

* It makes it impossible to express operations such as 
AudioBuffer.getChannelData() in JS terms.
* Since the interfaces are exposed to JS, the /only/ way to make them 
JS-friendly would be to change/extend the JS memory model (which is a 
much bigger task than to change the Web Audio API model).
* It's not in any way necessary for achieving the goal "glitch-free, 
reasonably-low-latency audio".

In fact, I believe it should be fully possible to implement the Web 
Audio API without using shared /mutable/ data internally (provided that 
we drop the shared data model from the interfaces). Correct me if I'm 
wrong, but I'm pretty sure you could solve most things by passing 
read-only references to data buffers between threads (which would be 
semantically equivalent to passing copies around) and still have the 
same memory/speed/latency performance. That would make it a moot point 
whether or not shared data must be part of a potential fictional 
execution environment or not.

/Marcus


>
>
> On Mon, Aug 5, 2013 at 1:49 PM, Alex Russell <slightlyoff@google.com 
> <mailto:slightlyoff@google.com>> wrote:
>
>     On Sun, Aug 4, 2013 at 6:46 PM, Chris Wilson <cwilso@google.com
>     <mailto:cwilso@google.com>> wrote:
>
>         (Sorry, on vacation, and beach > Web Audio discussions. :)
>
>         Alex, as we discussed a few weeks ago - glitch-free,
>         reasonably-low-latency audio is something that I just don't
>         believe JS - /as it is today - /can do.
>
>
>     In the TAG feedback document you might have detected two lines of
>     attack on this argument:
>
>      1. It's insufficient to say "JavaScript can't do this". The
>         entire goal of some of the suggestions in the document related
>         to the execution model are all about creating space in the
>         model to change the constraints under which JS (or something
>         else) runs such that JS//(or ASMJS or NaCL in a worker, etc.)
>         /could/ do what's needed without breaking the invariants of
>         either the platform or the conceptual semantic model that Web
>         Audio presents. So that's simply an argument against an
>         assertion /that hasn't been presented/. It fails on that
>         ground alone, however...
>      2. Nothing in my argument against breaking the platform
>         invariants requires that you actually RUN the built-in node
>         types in JS. To some great degree, I'm advocating for
>         JS-as-spec-language when I discuss de-sugaring. Not an
>         implementation strategy (except perhaps in naive cases for
>         newly-bootstrapping impls).
>
>     The net is that discussing main-thread-JS-as-we-know-it-today and
>     not the framework for execution/specification is arguing against
>     an implementation-level straw-man.
>
>     Regards
>
>         On Thu, Aug 1, 2013 at 5:42 PM, Alex Russell
>         <slightlyoff@google.com <mailto:slightlyoff@google.com>> wrote:
>
>             Let me be clearer, then: the issue is that it introduces
>             effects to JS that can't be described /in terms of JS/. It
>             violates the run-to-completion model of the language
>             without appealing to the turn-based concurrency model we
>             use everywhere else in the platform.
>
>
>             On Wed, Jul 31, 2013 at 10:43 AM, Chris Wilson
>             <cwilso@google.com <mailto:cwilso@google.com>> wrote:
>
>                 In addition, I'd ask that you be more explicit than
>                 calling this problem "data races", because there's
>                 clearly some explicit effect you're trying to prevent.
>                  Any asynchronously-programmable or event-driven
>                 system can enable developers to introduce race conditions.
>
>
>                 On Mon, Jul 29, 2013 at 10:09 PM, Noah Mendelsohn
>                 <nrm@arcanedomain.com <mailto:nrm@arcanedomain.com>>
>                 wrote:
>
>
>
>                     On 7/29/2013 7:05 PM, Anne van Kesteren wrote:
>
>                         On Sat, Jul 27, 2013 at 9:22 AM, Noah
>                         Mendelsohn<nrm@arcanedomain.com
>                         <mailto:nrm@arcanedomain.com>>  wrote:
>
>                             >Again, I have no informed opinions on the
>                             specific merits, just suggesting a
>                             >useful role the TAG might play to clarify
>                             for the many members of the
>                             >community who are less expert on this
>                             than you are. Thank you.
>
>
>                         I'm not sure we call out data races anywhere,
>                         it's something we just don't do.
>
>
>                     Well, my recollection may be faulty, but I think
>                     that one of the reasons the TAG took the trouble
>                     to formalize things like the architecture document
>                     was the belief that it's easier to ask skeptics to
>                     stick to rules that have been written down, and
>                     especially those that have garnered formal
>                     consensus through something like the
>                     Recommendation track.
>
>                     Whether it's worth taking a guideline on data
>                     races all the way to Rec I'm not sure, but it
>                     seems that it would be worth setting it down
>                     formally, perhaps in a TAG Finding/blog
>                     post/Recommendation or whatever will get the right
>                     level of discussion, consensus building, and
>                     eventually attention.
>
>                     Certainly, of the many things that have come up
>                     recently relating to APIs, this one seems deeply
>                     architectural and very much within the TAG's remit.
>
>                     Noah
>
>
>
>
>
>


-- 
Marcus Geelnard
Technical Lead, Mobile Infrastructure
Opera Software