W3C home > Mailing lists > Public > public-audio@w3.org > July to September 2014

Re: Looking for a method to provide a 5+ second waveform preview as a user plays a <video>

From: Eric Wescott <wescotte@gmail.com>
Date: Tue, 23 Sep 2014 18:07:26 -0500
Message-ID: <5421FD2E.3080202@gmail.com>
To: Raymond Toy <rtoy@google.com>
CC: "public-audio@w3.org" <public-audio@w3.org>
>
> Why not have the ScriptProcessor gather up about 20 of these 16k 
> buffers and then plot them all at once? Or just plot each 16k at the 
> appropriate point in your graph?
I don't quite understand how that would work... My understanding is 
"audioprocess" events dispatch at regular intervals by a ScriptProcessor 
and when I attempt to manually dispatch it via

var event = new Event('audioprocess');
for (var i=0; i < 20; i++)
     audioScript.dispatchEvent(event);

I get "Uncaught TypeError: Cannot read property 'getChannelData' of 
undefined" 20 or out 21 times from my function that handles 
"audioprocess" events.

However, even if that did work doesn't it just give you 16k samples 
based on the current audio being played? So if I dispatched 20 events in 
quick sucession would I just get 20 16 blocks with mostly overlapping 
sample data?

Could you elaborate?
>
>
>     I've also experimented with creating the entire waveform up front
>     <http://jsfiddle.net/daofw1g3/8/> which works but unfortunately
>     has issues with large files or longer audio clips.  I found if the
>     video file is > 1GB the FIle API dispatches an "error" event when
>     I readAsArrayBuffer(). So I thought perhaps I have to have the
>     user isolate the audio from the but that has it's own issues as
>     well.  I tested it with a 80minute 35mb mp3 and while ArrayBuffer
>     is created but then decodeAudioData() never runs it's callback
>     function. I assume both cases involve a memory limit?
>
>
> Well, 80 min of audio at 44.1 is like 800 MB of data. decodeAudioData 
> should work, but it may take some time to decode it into memory. 
> Assuming you have enough memory and process space.
>
I dunno I'm pretty convinced it's hitting some sort of memory limit. 
After file API creates the arrayBuffer my activity monitor shows no CPU, 
memory, or disk usage that suggests anything is actually going on and I 
let it sit for over a half hour and the callback is never executed...

> If you don't want to do that, perhaps using a MediaElementAudioSource 
> will work for you. It will stream the data into the audio graph where 
> you can process it without having it all in memory.
>
Could you elaborate on this? Because I think that is what I'm already 
doing by way the below line...

audioSource = 
audioContext.createMediaElementSource(document.getElementById("video"));


Thank you for your patients as I'm still pretty new to javascript and 
the Web Audio API
Eric
Received on Tuesday, 23 September 2014 23:07:55 UTC

This archive was generated by hypermail 2.3.1 : Tuesday, 6 January 2015 21:50:14 UTC