Re: New proposal for fixing race conditions

If realtime audio editing for large high bitrate audio files on memory
constrained devices without the use of swap/temporary storage is such an
important use case, maybe there should be actual test cases against which
an implementation like Firefox's can be tested, and then instead of
continuing to argue on hypotheticals we can look at the actual memory usage
overhead from the copies and CPU time overhead from calls to memcpy and
actually make an informed decision.

I feel like a broken record here continuing to repeat the fact that we have
no data based upon which we can argue that performance blocks any
particular design decision. The fact that new strawman use cases keep
getting introduced without remedying this problem is a little bewildering
to me. If the cost of building test cases to support your arguments and
then gathering data is too severe, we could look at all the major Web Audio
applications out there in the wild and see if the additional copies/memory
usage in FF's implementation causes them to perform worse.

You say you've made enormous efforts at optimizing Web Audio, so I assume
you must have done this optimization work based on actual profiling and
measurement of real applications or test cases, not just based on what you
believe to be slow. When I think of the challenges involved in running
applications and games on mobile devices in HTML5 right now, audio is not
even near the top of the list, because I've run into dozens of other issues
that are far more pressing - for other people to understand which
challenges you're referring to and why they're important, we need to see
those test cases. This conversation has been going in circles for a while
now; actual data might actually put an end to that. I don't think you're
going to convince people who are fervently anti-race-condition just by
asserting that the costs are too severe.


On Thu, Jul 25, 2013 at 3:15 PM, Chris Rogers <crogers@google.com> wrote:

>
>
>
> On Thu, Jul 25, 2013 at 1:45 PM, Ehsan Akhgari <ehsan.akhgari@gmail.com>wrote:
>
>> On Thu, Jul 25, 2013 at 4:30 PM, Chris Rogers <crogers@google.com> wrote:
>>
>>>
>>>
>>>
>>> On Thu, Jul 25, 2013 at 12:56 PM, Marcus Geelnard <mage@opera.com>wrote:
>>>
>>>> Hi Olivier,
>>>>
>>>>
>>>> On Wed, Jul 24, 2013 at 1:02 PM, Olivier Thereaux <
>>>> Olivier.Thereaux@bbc.co.uk> wrote:
>>>>
>>>>> Hi all,
>>>>>
>>>>> On 23/07/2013 17:11, "Chris Wilson" <cwilso@google.com> wrote:
>>>>>
>>>>> * the cost of the proposed solutions (e.g "I doubt that memcpy will do
>>>>> much harm" versus "quite a heavy weight on interacting with audio
>>>>> buffer
>>>>> data").
>>>>>
>>>>>
>>>>>
>>>> I would like to see some more facts on the table here too.
>>>>
>>>> I hope that I have shown that:
>>>>
>>>> 1) The speed impact of memcpy is negligible.
>>>>
>>>> 2) For the use cases I've considered so far the additional memory
>>>> impact is temporary and transient, and shouldn't prevent any use case.
>>>>
>>>> I think that any concrete examples showing the opposite would be
>>>> valuable to this discussion.
>>>>
>>>
>>> As I've mentioned before, it's not the cost of the memcpy() itself, but
>>> the memory footprint.
>>>
>>> PCM audio data in an AudioBuffer is 20Mb per minute for stereo @44.1KHz.
>>>  The cost is more than double for professional quality 96Khz data.
>>>
>>> In an audio analysis or editor application, it wouldn't be at all
>>> unusual to examine or display minutes of audio at a time.  Typical songs
>>> are 2 - 4 minutes long, and it wouldn't be unusual to deal with editing
>>> multiple songs at a time.  Even with a total length of audio data of only
>>> 10 minutes, we're already at 200Mb for the contents of the data.  Now think
>>> about what happens when it's necessary to create copies of that data, and
>>> we're now dealing with 400Mb!  Mobile devices are quite memory constrained,
>>> with even some relatively recent devices such as the iPhone 4 and similar
>>> Android devices having only 500Mb total RAM.  Keep in mind that not all of
>>> this memory will be available for audio data, since much of it is required
>>> for the resident system software, and other applications.
>>>
>>
>> That's true, but there are two things to keep in mind.
>>
>> 1.  None of the presented solution proposals make it necessary to perform
>> a memcpy in all cases.
>> 2.  Such audio editing application need to handle the case where they
>> don't have enough system memory at any rate in a graceful way.  There are
>> many devices out there which can run modern web browsers but still have
>> 256MB of RAM, which means that even in the current world, this application
>> cannot keep the entire content of the song in memory at all times.  Such
>> solutions can involve things such as storing PCM content in Indexed DB and
>> only load parts of it into memory for processing at a given point in time.
>>
>
> We don't want to exacerbate an already difficult situation on these
> devices.  You can't just say that these devices are low on memory already,
> so why don't we waste a large amount more memory causing very visible
> performance degradations!  You're creating many more opportunities for
> real-world troubles in this area, and we know this will be this case.  And
> all of this in order to address something which is a non-issue in the
> real-world.
>
>
>>   If we want to focus on this problem, there are a whole number of
>> improvements which we can make to the API to make such kinds of
>> applications possible on low to mid range mobile devices.
>>
>
>
> We have made enormous efforts at optimizing the code in WebKit/Blink for
> both desktop and mobile devices. Given the constraints on the use cases
> we'd like to solve, I think we've done a pretty good job so far.
>
>
>
>
>
>>
>>
>>> Unless we're talking about memory leaks, all memory allocations are
>>> temporary or transient, but their effect on system performance is not, with
>>> VM system swapping entering into the picture, degrading the entire device's
>>> performance.
>>>
>>
>> True.  On many of these devices there is no swap partition available at
>> all, even.
>>
>> Cheers,
>> --
>> Ehsan
>> <http://ehsanakhgari.org/>
>>
>>
>>>  Chris
>>>
>>>
>>>>
>>>>
>>>> /Marcus
>>>>
>>>>
>>>>
>>>>> Olivier
>>>>>
>>>>>
>>>>>
>>>>> -----------------------------
>>>>> http://www.bbc.co.uk
>>>>> This e-mail (and any attachments) is confidential and
>>>>> may contain personal views which are not the views of the BBC unless
>>>>> specifically stated.
>>>>> If you have received it in
>>>>> error, please delete it from your system.
>>>>> Do not use, copy or disclose the
>>>>> information in any way nor act in reliance on it and notify the sender
>>>>> immediately.
>>>>> Please note that the BBC monitors e-mails
>>>>> sent or received.
>>>>> Further communication will signify your consent to
>>>>> this.
>>>>> -----------------------------
>>>>>
>>>>
>>>>
>>>
>>
>