OfflineAudioContext specification gaps

Hi all,

I thought I would offer a few thoughts on OfflineAudioContext (OAC) to help identify areas that are not fully specified. My approach here is to ask: what are the observable aspects of a regular AudioContext (AC)'s state with respect to time, and how are those manifested in the offline case?

I've offered some initial suggestions on how to approach these issues to stimulate discussion. They are not exact enough to use in the spec and I apologize in advance for their lack of clarity. I hope these are helpful in advancing our understanding.

----

Issue: What is the overall algorithmic description of an OAC's rendering process?

Suggestion: Prior to the start of OAC rendering, AudioNodes connected to it "do nothing" (that is, they experience no forward motion of performance time). Once an OAC begins rendering, the AudioNode graph upstream processes audio exactly as if a regular AC's performance time was moving forward monotonically, starting from a value of zero.  The performance time value of zero (with respect to AudioProcessingEvent.playbackTime, source start/stop times and AudioParam value-curve times) is mapped to the first sample frame in the audio output emitted by the OAC. Upon reaching the limit of the supplied length argument in the constructor, the rendering process ends and performance time does not move forward any more.

----

Issue: Can an OAC be used to render more than one audio result?

Suggestion: No, it is a one-shot-use object (although it could render and deliver a single audio result in discrete chunks).

----

Issue: A regular AC's currentTime attribute progresses monotonically in lock-step with real time. What value does an OAC's currentTime present during the asynchronous rendering process?

Suggestion: Upon calling startRendering(), the currentTime value becomes zero. During rendering the currentTime attribute of an OAC MAY increase monotonically to approximately reflect the progress of the rendering process, whose rate may be faster or slower than real time. But whenever any rendering-related event is dispatched (e.g. oncomplete or any future incremental rendering event), the currentTime value MUST reflect the exact duration of all rendered audio up to that point.

----

Issue: It is not clear whether one can modify the node graph feeding an OAC. However, synthesis graphs feeding a real-time AC's destination are typically constructed in a just-in-time fashion driven by window.setInterval(), including only source nodes which are scheduled in a reasonably short time window into the future (e.g. 5-10 seconds). Thus, graphs feeding a real time AC need never become all that large and the work of constructing these graphs can be broken into nice processing slices.

Another way of saying this is that in an OAC there is no way to "granulate" the rendering process (at least, as long as we keep the approach that a single chunk of data is to be produced at the end). Thus, it seems that developers must assemble a single huge graph for the entire timespan to be rendered, at once. This is likely to tie up the main thread while application JS code constructs this huge graph.

Suggestion: Dispatch periodic "rendering partially complete" events from an OAC for reasonably sized chunks of data. Typically these would be much larger than 128-frame blocks; they would be in a multi-second timeframe. During handling of these events (but at no other times), AudioNodes may be removed from or added to the OAC's graph. This not only solves the issue detailed above, but also handles arbitrarily long audio output streams.  Note that one cannot easily use a sequence of multiple OACs on successive time ranges to simulate this outcome because of effect tail times.

Corollary: The consequences of modifying the graph of AudioNodes feeding the OAC during rendering are not defined EXCEPT when these modifications take place during these proposed events.

----

Issue: The spatialization attributes (location, orientation, etc.) of AudioListener and PannerNode cannot be scheduled. In a regular AC these can be modified in real time during rendering (I think). However, there is no way in an OAC to perform the same modifications at various moments in offline performance time.

Suggestion: Introduce events that trigger at given performance time offsets in an OAC? Replace these spatialization attributes with AudioParams? Simply stipulate that this can't be done?

----


.            .       .    .  . ...Joe

Joe Berkovitz
President

Noteflight LLC
Boston, Mass.
phone: +1 978 314 6271
www.noteflight.com
"Your music, everywhere"