Re: New proposal for fixing race conditions

On Thursday, July 25, 2013, Ehsan Akhgari <ehsan.akhgari@gmail.com> wrote:
> On Thu, Jul 25, 2013 at 4:30 PM, Chris Rogers <crogers@google.com> wrote:
>>
>>
>>
>> On Thu, Jul 25, 2013 at 12:56 PM, Marcus Geelnard <mage@opera.com> wrote:
>>>
>>> Hi Olivier,
>>>
>>>
>>> On Wed, Jul 24, 2013 at 1:02 PM, Olivier Thereaux <
Olivier.Thereaux@bbc.co.uk> wrote:
>>>>
>>>> Hi all,
>>>>
>>>> On 23/07/2013 17:11, "Chris Wilson" <cwilso@google.com> wrote:
>>>>
>>>> * the cost of the proposed solutions (e.g "I doubt that memcpy will do
>>>> much harm" versus "quite a heavy weight on interacting with audio
buffer
>>>> data").
>>>>
>>>>
>>>
>>> I would like to see some more facts on the table here too.
>>>
>>> I hope that I have shown that:
>>>
>>> 1) The speed impact of memcpy is negligible.
>>>
>>> 2) For the use cases I've considered so far the additional memory
impact is temporary and transient, and shouldn't prevent any use case.
>>>
>>> I think that any concrete examples showing the opposite would be
valuable to this discussion.
>>
>> As I've mentioned before, it's not the cost of the memcpy() itself, but
the memory footprint.
>> PCM audio data in an AudioBuffer is 20Mb per minute for stereo @44.1KHz.
 The cost is more than double for professional quality 96Khz data.
>> In an audio analysis or editor application, it wouldn't be at all
unusual to examine or display minutes of audio at a time.  Typical songs
are 2 - 4 minutes long, and it wouldn't be unusual to deal with editing
multiple songs at a time.  Even with a total length of audio data of only
10 minutes, we're already at 200Mb for the contents of the data.  Now think
about what happens when it's necessary to create copies of that data, and
we're now dealing with 400Mb!  Mobile devices are quite memory constrained,
with even some relatively recent devices such as the iPhone 4 and similar
Android devices having only 500Mb total RAM.  Keep in mind that not all of
this memory will be available for audio data, since much of it is required
for the resident system software, and other applications.
>
> That's true, but there are two things to keep in mind.
>
> 1.  None of the presented solution proposals make it necessary to perform
a memcpy in all cases.
> 2.  Such audio editing application need to handle the case where they
don't have enough system memory at any rate in a graceful way.  There are
many devices out there which can run modern web browsers but still have
256MB of RAM, which means that even in the current world, this application
cannot keep the entire content of the song in memory at all times.  Such
solutions can involve things such as storing PCM content in Indexed DB and
only load parts of it into memory for processing at a given point in time.
If we want to focus on this problem, there are a whole number of
improvements which we can make to the API to make such kinds of
applications possible on low to mid range mobile devices.
>

I think you have a point here. In general I'm a sucker for solutions that
save cycles and bytes, but for making sound editing scalable across devices
you need to have a method that gracefully degrades for lower end machines
anyway.

I'd probably attack it by processing data in chunks, possibly 1-10 MB in
size. And then there's the whole issue of displaying multi-MB data on
screen in 60 fps regardless of zoom-level, which you can't do by accessing
the raw PCM data directly anyway (you need clever data structures).

In any event, if we want to squeeze every byte out for speeding up editing
of multi-MB audio data, there must be lots of things that can be done, but
I think it's better to do it for a later version. The data race issue is
harder to fix later though (adding new functionality is easier than to
remove old functionality).

/Marcus

>>
>> Unless we're talking about memory leaks, all memory allocations are
temporary or transient, but their effect on system performance is not, with
VM system swapping entering into the picture, degrading the entire device's
performance.
>
> True.  On many of these devices there is no swap partition available at
all, even.
>
> Cheers,
> --
> Ehsan
> <http://ehsanakhgari.org/>
>
>>
>> Chris
>>
>>>
>>> /Marcus
>>>
>>>
>>>>
>>>> Olivier
>>>>
>>>>
>>>>
>>>> -----------------------------
>>>> http://www.bbc.co.uk
>>>> This e-mail (and any attachments) is confidential and
>>>> may contain personal views which are not the views of the BBC unless
specifically stated.
>>>> If you have received it in
>>>> error, please delete it from your system.
>>>> Do not use, copy or disclose the
>>>> information in any way nor act in reliance on it and notify the sender
>>>> immediately.
>>>> Please note that the BBC monitors e-mails
>>>> sent or received.
>>>> Further communication will signify your consent to
>>>> this.
>>>> -----------------------------
>>>
>>
>
>