RE: approaches to recording

Implicitly, at least, there are two APIs: one for streaming and one for file-level recording.  If we're doing streaming recording on a MediaStream, and new Tracks get added, do we ignore them or signal an error?  What do we do if Tracks get removed?  Signal an error or fill in with black frames/silence?  

- Jim

-----Original Message-----
From: Timothy B. Terriberry [mailto:tterriberry@mozilla.com] 
Sent: Wednesday, October 17, 2012 8:39 AM
To: public-media-capture@w3.org
Subject: Re: approaches to recording

Jim Barnett wrote:
> Are there encodings that can handle dynamically adding tracks that do _not_ support streaming encoding?  Streaming encoding may well belong at the Track level, anyway.  I think that it's  primary purpose is to support media processing (including  speech recognition), and for those uses you want to get pure audio, pure video, etc.

If you're not doing streaming encoding, you don't have to "dynamically" 
add tracks. You can simply collect all the data (possibly caching encoded versions on disk, but that is an implementation detail) and mux the results at the end, when you're done.

You will probably need to insert black frames / silence for tracks that do not begin at the start of the stream, as most playback software will be confused if there is no data for them for a track for an extended period of time. Likely also at the end, too (if you want seeking to work).

However, I agree with others that a separate "track-level" recording interface is a bad idea. There's nothing it can do that can't be done by a stream-level interface using a stream with a single track, but there are plenty of things a track-level interface cannot do that a stream-level interface can (e.g., synchronization). This issue is completely orthogonal to whether you want an interface that returns an entire encoded file at once, or whether you want to support streaming encoding. I think it's a mistake to conflate the two, as there are plenty of uses for streaming encoding that involve synchronized audio and video (our weekly Air Mozilla broadcasts being one, just for example).

Received on Wednesday, 17 October 2012 15:01:41 UTC