Re: Web Audio API decoder instancing

Hello Adam,

On Fri, Dec 21, 2012 at 11:51 AM, Bradley, Adam <adam.bradley@jagex.com>wrote:

> Well I looked into <audio> elements with some interest, however as far as
> I could figure out, you can’t define their source content using a buffer of
> data (an ArrayBuffer for example), and instead they require a URL to a
> source file that they then buffer themselves. This may not be enough
> control for my purposes, especially as from what I can make out about
> browser support, that in some cases I cannot use multiple <audio> elements
> concurrently, which will not be satisfactory for our needs. We also like to
> pack our audio data into custom obfuscated formats rather than to use loose
> files, which the <audio> element system doesn’t seem capable of integrating
> with.
>

This is correct.


> Is there another way of achieving what I’m after?
>

You might want to take a look at the audio decoder suite, aurora.js (
https://github.com/ofmlabs/aurora.js ). One of its goals is to be able to
handle 1h+ podcasts, so streaming is something we've supported from day one
and it has worked well so far. However, there's no codec for .ogg yet
(we've covered mostly codecs that browsers don't have native support for so
far, and now have ALAC, FLAC, AAC and MP3 and of course the basic
uncompressed formats. Obviously, all of it is written in JS so it's unable
to take advantage of hardware decoders or such, but it has a fairly low CPU
impact (my laptop runs the decoders underclocking at 800MHz, using about
10% CPU), and we even have parallelization in the works (works already, but
needs some optimization). For your use case, I can't tell if it's better to
make a custom demuxer or a custom streaming source.


> I would also like more information, if it is available, as to why the
> decoding system works in the way it does, what is the reasoning behind
> implementing a global decoder rather than instancing? It just seems like it
> would be far more flexible if it could be instanced, and would eat far less
> memory. Am I missing something about the implementation here?
>

Chris will know better, but I think this is just because of the planned use
case, i.e. unpacking a short sample to be used as a BufferSourceNode or
impulse response. For that use case the current form is probably the
simplest, but I agree that a stateful decoder would be orders of magnitude
more useful.

Cheers,
Jussi


>  **
>
> Cheers,****
>
> ** **
>
> *From:* Chris Rogers [mailto:crogers@google.com]
> *Sent:* 20 December 2012 22:49
> *To:* Bradley, Adam
> *Cc:* public-audio@w3.org
> *Subject:* Re: Web Audio API decoder instancing****
>
> ** **
>
> ** **
>
> On Thu, Dec 20, 2012 at 3:23 AM, Bradley, Adam <adam.bradley@jagex.com>
> wrote:****
>
> Hi,****
>
>  ****
>
> I’m interested in finding out more information on the web audio API
> standard with regards to decoding encoded formats (Ogg/Mp3 etc.).****
>
>  ****
>
> What I am wondering is why the standard only allows for a ‘global’ decoder
> that accepts an intact audio stream, rather than being able to instance a
> decoder object and then pass the audio data to it as and when the
> javascript application decides in chunks.****
>
>  ****
>
> The reason I’m interested in this is that I am wanting to get large
> (>10min) audio files in Ogg format playing back reliably in HTML5 whilst
> using as little memory as possible. It seems, at least naively to me, that
> this would be handled better if we could decode the files in chunks using
> an instanced decoder and thus employ some manner of circular buffering
> technique to keep the memory requirements to a minimum.****
>
>  ****
>
> Is this something that may be possible to build into the standard, or
> perhaps there is another existing solution to this issue that I am
> overlooking?****
>
> ** **
>
> What you might be interested in is using an <audio> element for streaming
> and using the MediaElementAudioSourceNode:****
>
>
> https://dvcs.w3.org/hg/audio/raw-file/tip/webaudio/specification.html#MediaElementAudioSourceNode
> ****
>
>  ****
>
>  ****
>
> Cheers,****
>
>  ****
>
> *Adam Bradley*****
>
> Audio Systems Developer****
>
> Jagex Games Studio****
>
>  ****
>
> -- ****
>
> IMPORTANT NOTICE****
>
> The sender does not guarantee that this message, including any attachment,
> is secure or virus free. Also, it is confidential and may be privileged or
> otherwise protected from disclosure. If you are not the intended recipient,
> do not disclose or copy it or its contents. Please telephone or email the
> sender and delete the message entirely from your system. No binding
> obligations or payment commitments are to be derived from the contents of
> this email unless and until a clear written agreement containing all the
> necessary terms and conditions is properly executed.****
>
> Jagex Limited is a company registered in England & Wales with company
> number 03982706 and a registered office at St John's Innovation Centre,
> Cowley Road, Cambridge, CB4 0WS, UK.****
>
>  ****
>
> ** **
>
> --
>
> IMPORTANT NOTICE
>
> The sender does not guarantee that this message, including any attachment,
> is secure or virus free. Also, it is confidential and may be privileged or
> otherwise protected from disclosure. If you are not the intended recipient,
> do not disclose or copy it or its contents. Please telephone or email the
> sender and delete the message entirely from your system. No binding
> obligations or payment commitments are to be derived from the contents of
> this email unless and until a clear written agreement containing all the
> necessary terms and conditions is properly executed.
>
> Jagex Limited is a company registered in England & Wales with company
> number 03982706 and a registered office at St John's Innovation Centre,
> Cowley Road, Cambridge, CB4 0WS, UK.
>
>
>