MediaRecorder and using Streams

Greg Billock and I have been taking a look at the MediaRecorder API for
Chromium, and we were wondering if there’s a reason MediaRecorder doesn’t
use the Streams API. Intuitively, it seems to make sense, since the end
product is an encoded video stream.


   -

   There seem to be two different use-cases, sliced recording and one-shot
   recording
   -

   These two cases face different resource constraints. Sliced recording is
   presumably used mostly for streaming purposes, and thus a bit more
   sensitive to resource constraints.
   -

   The current API can easily create an unbounded number of blobs.
   -

   The use of blobs in ondataavailable raises the interesting question of
   the proper mime type for each blob.
   -

   The possibility automatically switching to disk-backed blobs raises
   another interesting question, that of quota interaction.


Given these issues, we’ve looked at an alternative approach using Streams
instead. Specifically, the API would be restructured to allow the following
calls:

Stream recordToStream(optional StreamBuilder b);

void recordToBlob();

recordToBlob covers the use case of recording in one go - it will simply
have a finalized Blob available when onstop() is fired.

recordToStream() offers the ability for timesliced recording. It will build
a Stream populated with the encoded data, either using an internally
constructed StreamBuilder or using the passed-in StreamBuilder.

Here are the advantages as we see them:

1) It separates out the two use cases of sliced vs. non-sliced recording.
The UA is in full control of use of memory vs. disk storage for the
one-shot recording.

2) It is more resilient against accidental blob leakage. Instead of having
to decide for each blob if it should have disk backing or not, this is up
to the Stream implementation. Users are guided more naturally into not
creating code that queues up blobs that it fails to consume at sufficient
speed - this buffering is all more naturally left to the Stream object.

2a) That means the UA can implement “smart” streams that react accordingly
to resource constraints - discard data up to the next I-frame, replace
frames with small placeholder frames to indicate they were skipped, etc.

3) Users can still get more control over the creation of the stream if they
pass a StreamBuilder.

3a) A lot of the error handling and event processing move into the stock
File API object.

3b) A user-built StreamBuilder writing to long-term storage has more
predictable interaction with the quota system for disk-stored recordings
without needing to move the Blob resulting from one-shot recording.

4) There is no type mystery any more - there is exactly one definitive
type, and it is stored on the Stream. No need to deal with data fragments
of unknowable type.


If recordToStream is not given the optional StreamBuilder parameter, the UA
can use an internal implementation.

Would love to hear your thoughts,
 - rachel