Re: decodeAudioBuffer, XHR and MIME headers

Not to sidetrack, but 'streaming' is the key here. I've seen multiple
mentions on this list (in particular re: the race condition/memcpy
discussions) of the importance of being able to process large amounts of
audio, and in that scenario having to download an entire 50MB of audio into
a buffer before passing it into decodeAudioBuffer to decode all at once
into another buffer is suboptimal. I said 'streaming' because being able to
download and decode an MP3 in chunks of say 128kb at a time has lots of
benefits - reduced memory usage, reduced latency before playback can begin,
etc.

However, I absolutely had not considered that Blob has a mime-type baked
in. Based on that it seems like being able to pass a Blob directly into the
decoder would address all the basic use cases here.


On Wed, Jul 24, 2013 at 1:55 PM, Mandyam, Giridhar <mandyam@quicinc.com>wrote:

>  > Given that there is currently no standard for streaming binary XHR, it
> is definitely unfortunate that on top of that decodeAudioBuffer is so rigid.
> ****
>
> ** **
>
> I’m missing something – XHR2 allows for transmission of blobs (
> http://www.w3.org/TR/XMLHttpRequest2/#interface-xmlhttprequest), and the
> blobs would presumably have accurately set the content type and ergo it
> would be available via the blob.  To me that satisfies any requirement for
> delivery of binary data via XHR and providing the content type to user
> space.  What is wrong with my interpretation?****
>
> ** **
>
> Regardless, XHR is probably not really relevant to the core of Joe’s
> concern – decodeAudioData still would have to discern the MIME type from
> the ArrayBuffer given the current API design.****
>
> ** **
>
> -Giri Mandyam, Qualcomm Innovation Center****
>
> ** **
>
> ** **
>
> *From:* kevin.gadd@gmail.com [mailto:kevin.gadd@gmail.com] *On Behalf Of *K.
> Gadd
> *Sent:* Wednesday, July 24, 2013 12:19 PM
> *To:* Joseph Berkovitz
> *Cc:* public-audio@w3.org WG
> *Subject:* Re: decodeAudioBuffer, XHR and MIME headers****
>
> ** **
>
> The lack of MIME type information has in fact already caused lots of
> problems for applications, due to partial/broken type sniffing for flexible
> formats like MP3. There are some open bugs against FF's implementation
> still (I hit one of them; it looks like people have hit others)****
>
> Given that there is currently no standard for streaming binary XHR, it is
> definitely unfortunate that on top of that decodeAudioBuffer is so rigid.
> Some sort of mechanism for either fetching decoded audio from a URI (in one
> go) or for streaming chunks of decoded audio as it comes in (a sort of
> 'request audio from URI; events periodically fire providing decoded chunks
> once they are ready' thing) would be welcomed even if it's not necessary.
> Right now I have a bunch of code dedicated to handling this problem myself
> in a cross-browser fashion, and it eats a lot of extra RAM and CPU cycles
> for temporary copies and sniffing and such.****
>
> ** **
>
> On Wed, Jul 24, 2013 at 11:55 AM, Joseph Berkovitz <joe@noteflight.com>
> wrote:****
>
> Prompted by something Jussi said this morning, I wonder if
> decodeAudioBuffer() doesn't have some other built-in disadvantages for
> processing XHRs besides the extra buffer copying overhead that he pointed
> out:****
>
> ** **
>
> - XHR response headers for the content MIME type are not available to the
> decoding process, so the data alone must supply information on what what
> codec to use. This may not be an issue to implementors, but I believe that
> the <audio> tag is sensitive to MIME types in some cases, and I wonder why
> decodeAudioBuffer() would be any less interested in that information.****
>
> ** **
>
> - There is no possibility of decoding incoming data on a pipelined basis
> as it arrives.****
>
> ** **
>
> I do not suggest we perturb the current API. I am just wondering if it is
> worth at some point introducing an additional decodeAudioURI() call that
> takes a URI, not an ArrayBuffer. I will pass on the debate over whether it
> should return an AudioBuffer or, as Jussi suggested, audio data in typed
> arrays.****
>
> ** **
>
> Best,****
>
> ** **
>
> .            .       .    .  . ...Joe****
>
> ** **
>
> *Joe Berkovitz*****
>
> President****
>
> ** **
>
> *Noteflight LLC*****
>
> Boston, Mass.****
>
> phone: +1 978 314 6271****
>
> www.noteflight.com****
>
> "Your music, everywhere"****
>
> ** **
>
> ** **
>