RE: decodeAudioBuffer, XHR and MIME headers

> Given that there is currently no standard for streaming binary XHR, it is definitely unfortunate that on top of that decodeAudioBuffer is so rigid.

I’m missing something – XHR2 allows for transmission of blobs (http://www.w3.org/TR/XMLHttpRequest2/#interface-xmlhttprequest), and the blobs would presumably have accurately set the content type and ergo it would be available via the blob.  To me that satisfies any requirement for delivery of binary data via XHR and providing the content type to user space.  What is wrong with my interpretation?

Regardless, XHR is probably not really relevant to the core of Joe’s concern – decodeAudioData still would have to discern the MIME type from the ArrayBuffer given the current API design.

-Giri Mandyam, Qualcomm Innovation Center


From: kevin.gadd@gmail.com [mailto:kevin.gadd@gmail.com] On Behalf Of K. Gadd
Sent: Wednesday, July 24, 2013 12:19 PM
To: Joseph Berkovitz
Cc: public-audio@w3.org WG
Subject: Re: decodeAudioBuffer, XHR and MIME headers

The lack of MIME type information has in fact already caused lots of problems for applications, due to partial/broken type sniffing for flexible formats like MP3. There are some open bugs against FF's implementation still (I hit one of them; it looks like people have hit others)
Given that there is currently no standard for streaming binary XHR, it is definitely unfortunate that on top of that decodeAudioBuffer is so rigid. Some sort of mechanism for either fetching decoded audio from a URI (in one go) or for streaming chunks of decoded audio as it comes in (a sort of 'request audio from URI; events periodically fire providing decoded chunks once they are ready' thing) would be welcomed even if it's not necessary. Right now I have a bunch of code dedicated to handling this problem myself in a cross-browser fashion, and it eats a lot of extra RAM and CPU cycles for temporary copies and sniffing and such.

On Wed, Jul 24, 2013 at 11:55 AM, Joseph Berkovitz <joe@noteflight.com<mailto:joe@noteflight.com>> wrote:
Prompted by something Jussi said this morning, I wonder if decodeAudioBuffer() doesn't have some other built-in disadvantages for processing XHRs besides the extra buffer copying overhead that he pointed out:

- XHR response headers for the content MIME type are not available to the decoding process, so the data alone must supply information on what what codec to use. This may not be an issue to implementors, but I believe that the <audio> tag is sensitive to MIME types in some cases, and I wonder why decodeAudioBuffer() would be any less interested in that information.

- There is no possibility of decoding incoming data on a pipelined basis as it arrives.

I do not suggest we perturb the current API. I am just wondering if it is worth at some point introducing an additional decodeAudioURI() call that takes a URI, not an ArrayBuffer. I will pass on the debate over whether it should return an AudioBuffer or, as Jussi suggested, audio data in typed arrays.

Best,

.            .       .    .  . ...Joe

Joe Berkovitz
President

Noteflight LLC
Boston, Mass.
phone: +1 978 314 6271<tel:%2B1%20978%20314%206271>
www.noteflight.com<http://www.noteflight.com>
"Your music, everywhere"