- From: Marcus Geelnard <mage@opera.com>
- Date: Wed, 7 Aug 2013 08:50:06 +0200
- To: Chris Wilson <cwilso@google.com>
- Cc: Alex Russell <slightlyoff@google.com>, Noah Mendelsohn <nrm@arcanedomain.com>, Anne van Kesteren <annevk@annevk.nl>, Olivier Thereaux <Olivier.Thereaux@bbc.co.uk>, "robert@ocallahan.org" <robert@ocallahan.org>, "public-audio@w3.org" <public-audio@w3.org>, "www-tag@w3.org List" <www-tag@w3.org>
- Message-ID: <CAL8YEv4KW9o9ctQNF1-vNnsZp7YhmH6+GxKzQ=CHrLdPAXiWWw@mail.gmail.com>
2013/8/6 Chris Wilson <cwilso@google.com> > I still (obviously) disagree that a model that relies on passing copies > around will have the same memory/speed/latency performance. > Sorry, maybe I wasn't clear enough... I meant as an example that *internally* you could pass *references to read-only buffers* around (which has about zero overhead), in order to highlight the fact that sharing of mutable data between threads is *not* a requirement for implementing an audio API with good memory/speed/latency performance. It's possible the actual copying can be minimized, at a cost in usability > to the developer, but even with that I don't think the cost will ever be > truly zero; so this is a tradeoff. > Of course it's a trade off, just as using 32-bit floats instead of 64-bit floats or 16-bit fixed points was a tradeoff. In the end we aim to hit the "sweet spot". Regarding "cost in usability", that's a fairly subjective statement. I personally find the current API quite non-intuitive to work with, much because it uses lots of non-JS constructs (the getChannelData() method, for instance, works in a way that I've never seen on the Web before). > It may be that the group chooses that tradeoff - but traditional audio > APIs have not. > I guess that depends on what you mean with "traditional audio APIs". OpenAL, for instance, uses explicit copying. /Marcus > Of course, most of them don't have the limited execution scope of JS, > either. (Note that I should have said "glitch-free, reasonably-low-latency > audio with good performance and API usability." :) > > Interestingly enough, if we'd not allowed JS in nodes (rather used a more > restricted audio processing language), I'm not sure we'd have this problem > to this scale. > > > On Tue, Aug 6, 2013 at 2:43 AM, Marcus Geelnard <mage@opera.com> wrote: > >> 2013-08-06 04:20, Chris Wilson skrev: >> >> See, I read two different things from this. Marcus, I heard you say the >> comment about not being describable in JS was in specific reference to >> threading and memory sharing not being part of the current model; from >> Alex, I heard "open your mind to what JS *could* be, and then describe >> everything in one model" - in part, the nodes' behaviors themselves should >> be described in JS, albeit of course not necessarily run in JS. >> >> The latter (as I'd previously expressed to Alex) is fine with me; I'd >> be fine describing how delay nodes work in JS. However, high-quality >> glitch-free audio requires a number of things that aren't available to JS >> today - notably, a separate thread with restricted execution capabilities >> (i.e. not arbitrary JavaScript) and no garbage collection that can have >> elevated priority, and access to the physical hardware, of course. It's >> not at all impossible to imagine how to describe Web Audio if you have >> those capabilities; and the memory sharing we're discussing under the >> rubric of race conditions is really whether that somewhat different >> execution environment supports those or not. >> >> >> I have always been a strong supporter of being able to describe and >> implement as much as possible of the API in JS, and I agree that there are >> certainly some important aspects missing in the JS execution environment >> that make a practical implementation in JS impossible (including access to >> audio HW and low-level control over thread execution etc). You should, >> however, still be able to implement the *behavior* of the API in JS (as >> an exercise - imagine implementing an OfflineAudioContext in JS - I don't >> see why it shouldn't be possible). >> >> I might have misinterpreted Alex's point (I thought he was referring to >> the shared data in the API interfaces, but of course there may be more >> things to it). However, from the perspective of making the Web Audio API >> describable in JS terms, I still think that the issue of exposing shared >> data interfaces to JS is a key deficiency in the current design, mostly >> because: >> >> * It makes it impossible to express operations such as >> AudioBuffer.getChannelData() in JS terms. >> * Since the interfaces are exposed to JS, the *only* way to make them >> JS-friendly would be to change/extend the JS memory model (which is a much >> bigger task than to change the Web Audio API model). >> * It's not in any way necessary for achieving the goal "glitch-free, >> reasonably-low-latency audio". >> >> In fact, I believe it should be fully possible to implement the Web Audio >> API without using shared *mutable* data internally (provided that we >> drop the shared data model from the interfaces). Correct me if I'm wrong, >> but I'm pretty sure you could solve most things by passing read-only >> references to data buffers between threads (which would be semantically >> equivalent to passing copies around) and still have the same >> memory/speed/latency performance. That would make it a moot point whether >> or not shared data must be part of a potential fictional execution >> environment or not. >> >> /Marcus >> >> >> >> >> >> On Mon, Aug 5, 2013 at 1:49 PM, Alex Russell <slightlyoff@google.com>wrote: >> >>> On Sun, Aug 4, 2013 at 6:46 PM, Chris Wilson <cwilso@google.com> wrote: >>> >>>> (Sorry, on vacation, and beach > Web Audio discussions. :) >>>> >>>> Alex, as we discussed a few weeks ago - glitch-free, >>>> reasonably-low-latency audio is something that I just don't believe JS - >>>> *as it is today - *can do. >>>> >>> >>> In the TAG feedback document you might have detected two lines of >>> attack on this argument: >>> >>> >>> 1. It's insufficient to say "JavaScript can't do this". The entire >>> goal of some of the suggestions in the document related to the execution >>> model are all about creating space in the model to change the constraints >>> under which JS (or something else) runs such that JS* *(or ASMJS or >>> NaCL in a worker, etc.) *could* do what's needed without breaking >>> the invariants of either the platform or the conceptual semantic model that >>> Web Audio presents. So that's simply an argument against an assertion >>> *that hasn't been presented*. It fails on that ground alone, >>> however... >>> 2. Nothing in my argument against breaking the platform invariants >>> requires that you actually RUN the built-in node types in JS. To some great >>> degree, I'm advocating for JS-as-spec-language when I discuss de-sugaring. >>> Not an implementation strategy (except perhaps in naive cases for >>> newly-bootstrapping impls). >>> >>> The net is that discussing main-thread-JS-as-we-know-it-today and not >>> the framework for execution/specification is arguing against an >>> implementation-level straw-man. >>> >>> Regards >>> >>> On Thu, Aug 1, 2013 at 5:42 PM, Alex Russell <slightlyoff@google.com>wrote: >>>> >>>>> Let me be clearer, then: the issue is that it introduces effects to JS >>>>> that can't be described *in terms of JS*. It violates the >>>>> run-to-completion model of the language without appealing to the turn-based >>>>> concurrency model we use everywhere else in the platform. >>>>> >>>>> >>>>> On Wed, Jul 31, 2013 at 10:43 AM, Chris Wilson <cwilso@google.com>wrote: >>>>> >>>>>> In addition, I'd ask that you be more explicit than calling this >>>>>> problem "data races", because there's clearly some explicit effect you're >>>>>> trying to prevent. Any asynchronously-programmable or event-driven system >>>>>> can enable developers to introduce race conditions. >>>>>> >>>>>> >>>>>> On Mon, Jul 29, 2013 at 10:09 PM, Noah Mendelsohn < >>>>>> nrm@arcanedomain.com> wrote: >>>>>> >>>>>>> >>>>>>> >>>>>>> On 7/29/2013 7:05 PM, Anne van Kesteren wrote: >>>>>>> >>>>>>>> On Sat, Jul 27, 2013 at 9:22 AM, Noah Mendelsohn< >>>>>>>> nrm@arcanedomain.com> wrote: >>>>>>>> >>>>>>>>> >Again, I have no informed opinions on the specific merits, just >>>>>>>>> suggesting a >>>>>>>>> >useful role the TAG might play to clarify for the many members of >>>>>>>>> the >>>>>>>>> >community who are less expert on this than you are. Thank you. >>>>>>>>> >>>>>>>> >>>>>>> I'm not sure we call out data races anywhere, it's something we >>>>>>>> just don't do. >>>>>>>> >>>>>>> >>>>>>> Well, my recollection may be faulty, but I think that one of the >>>>>>> reasons the TAG took the trouble to formalize things like the architecture >>>>>>> document was the belief that it's easier to ask skeptics to stick to rules >>>>>>> that have been written down, and especially those that have garnered formal >>>>>>> consensus through something like the Recommendation track. >>>>>>> >>>>>>> Whether it's worth taking a guideline on data races all the way to >>>>>>> Rec I'm not sure, but it seems that it would be worth setting it down >>>>>>> formally, perhaps in a TAG Finding/blog post/Recommendation or whatever >>>>>>> will get the right level of discussion, consensus building, and eventually >>>>>>> attention. >>>>>>> >>>>>>> Certainly, of the many things that have come up recently relating to >>>>>>> APIs, this one seems deeply architectural and very much within the TAG's >>>>>>> remit. >>>>>>> >>>>>>> Noah >>>>>>> >>>>>>> >>>>>> >>>>> >>>> >>> >> >> >> -- >> Marcus Geelnard >> Technical Lead, Mobile Infrastructure >> Opera Software >> >> >
Received on Wednesday, 7 August 2013 06:50:35 UTC