Re: OfflineAudioContext specification gaps

On Mon, May 6, 2013 at 8:57 PM, Ehsan Akhgari <ehsan.akhgari@gmail.com>wrote:

> "Do nothing" needs to be clearly spec'ed.  Consider the following
> pseudo-code:
>
> var oc = new OfflineAudioContext(...);
> oc.oncomplete = function ocOnComplete() {};
> var sp = oc.createScriptProcessor();
> sp.onaudioprocess = function spOnAudioProcess() {
>   oc.startRendering();
>   sp.onaudioprocess = null;
> };
> sp.connect(oc.destination);
>
> Would ocOnComplete ever be called?
>

I think the intent is clearly that spOnAudioProcess would never be called,
and neither would ocOnComplete. It makes sense to me.

I think from the viewpoint of web authors, it is quite unreasonable to
> disallow modifications to the graph after
> OfflineAudioContext.startRendering is called, as that breaks one of the
> fundamental pieces of the processing model as defined elsewhere in the
> spec.  However, with my implementer's hat on, allowing such modifications
> would create all sorts of racing behavior, effects of which are clearly
> visible to web content.  Consider the following pseudo-code:
>
> var oc = new OfflineAudioContext(...);
> var source = oc.createBufferSource();
> source.buffer = ...;
> source.start(0);
> oc.startRendering();
> source.connect(destination); // Note: called *after* startRendering
> setTimeout(function() {
>   source.disconnect(oc.destination);
> }, 100);
>
> On a regular AudioContext, this will let the source node play back for
> about 100ms.  If the implementation does support faster than realtime
> processing for OfflineAudioContext, what would the renderedBuffer contain?
> It seems like the results of such processing would be different in two runs
> of this pseudo-code, which will provide an inherently unreliable API which
> doesn't have deterministic behavior from the viewpoint of the web content.
>

Having the graph change happen at an unpredictable time during processing
isn't that much worse than a regular AudioContext, where the graph changes
also happen at an unpredictable time. The range of uncertainty is probably
smaller for an AudioContext but it's exactly the same issue otherwise.

The idea that startRendering should take an optional duration, and the OAC
should render exactly that much data and then pause again, had a lot of
support. If authors ensure they only modify the OAC graph between receiving
a completion event and the next call to startRendering, results would be
deterministic.

Another big question that I have is how reasonable is it to expect the web
> author to know the desired length of the OfflineAudioContext's rendered
> buffer in advance?  This is especially worrying if, for example, you want
> to render the output of an HTMLMediaElement, for example.  The way that the
> spec is currently defined, if the media element doesn't have output
> immediately available for any reason, then you'll just get a number of
> silent frames in the beginning of the rendered buffer, and not all of the
> expected rendered frames.  This is an issue will all sorts of potential
> sources of delay in the graph that are not under the control of the web
> author.  That said, even for graphs which do not have any such sources of
> delays, it could still be quite tricky to compute the precise buffer length
> in advance, for example, in presence of loops in the graphs with DelayNodes
> that change their delays dynamically according to automation events
> scheduled on the delay AudioParam, etc.
>

Yeah, I don't think requiring authors to specify the total duration
up-front is reasonable. If we adopt the startRendering duration proposal,
authors can call startRendering repeatedly to render chunks until they
detect they want to stop.

Rob
-- 
q“qIqfq qyqoquq qlqoqvqeq qtqhqoqsqeq qwqhqoq qlqoqvqeq qyqoquq,q qwqhqaqtq
qcqrqeqdqiqtq qiqsq qtqhqaqtq qtqoq qyqoquq?q qEqvqeqnq qsqiqnqnqeqrqsq
qlqoqvqeq qtqhqoqsqeq qwqhqoq qlqoqvqeq qtqhqeqmq.q qAqnqdq qiqfq qyqoquq
qdqoq qgqoqoqdq qtqoq qtqhqoqsqeq qwqhqoq qaqrqeq qgqoqoqdq qtqoq qyqoquq,q
qwqhqaqtq qcqrqeqdqiqtq qiqsq qtqhqaqtq qtqoq qyqoquq?q qEqvqeqnq
qsqiqnqnqeqrqsq qdqoq qtqhqaqtq.q"