[mediacapture-record] Add method MediaRecorder.restart considering this use case. (#178)

titoBouzout has just created a new issue for https://github.com/w3c/mediacapture-record:

== Add method MediaRecorder.restart considering this use case. ==
Thanks for reading. I have been struggling to solve something very simple that I can't believe wasn't considered. 

I have a MediaStream which is recorded via MediaRecorder and that stream is sent to N clients. This works in a perfect world. 

In the real world clients just join and leave all the time middle stream. It looks like there's no way for a client to join a stream without restarting the stream, which will of course generate a glitch for every client of the mentioned stream.

I will try to explain with an example, for the sake of simplicity I left out irrelevant code.

**streamer.js**  --  So this page gets a stream and records it. This page never dies, the stream is infinite.

```javascript
// streamer.js

var recorder = new MediaRecorder(stream)
recorder.ondataavailable = function(e) {
 if (e.data.size > 0) {
  send_data_to_client_as_arraybuffer(e.data)
 }
}
recorder.start(1000)
```

**client.js**  --  So this page receives the blobs from streamer.js and plays it. There could be as many client.js as you need.

```javascript

// client.js

var media_source = new MediaSource()
media_source.onsourceopen = function (event) {
 var source_buffer = media_source.addSourceBuffer('video/webm;codecs=vp9')
 window.on_data_from_streamer = function(arraybuffer) {
  if (source_buffer.updating === false) {
   source_buffer.appendBuffer(new Uint8Array(arraybuffer))
  }
 }
}
var video = document.createElement('video')
video.src = URL.createObjectURL(media_source)
```

This code just works in a perfect world. So what's the problem with it? 

- a client.js can't just join if the streamer.js already started

Why? Because:

> The UA MUST record stream in such a way that the original Tracks can be retrieved at playback time. When multiple Blobs are returned (because of timeslice or requestData()), the individual Blobs need not be playable, but the combination of all the Blobs from a completed recording MUST be playable.

You can't just feed source_buffer.appendBuffer with blobs from the middle of a stream, it will not work.

I know I could just start a new parallel stream.js to serve the client that joined, but my bandwidth will not scale, I can't afford it. I also know about the canvas hack, I need a real solution.

Something I tried:

1. To start and stop the Stream in an interval to generate small videos. 

Problems with this:

1. ondataavailable gives you chunks that may not contain a complete last segment. So when you append this video via source_buffer.appendBuffer the buffer gets stuck and will not allow you to update the timestampOffset because is waiting for the remaining data to complete the segment parsing. What this means? You made a small video on which the last segment is incomplete. source_buffer.abort() will just drop that segment which means that you will be dropping segments each time you append a video, so the video and audio will glitch. Im not sure about the following, but as you have an incomplete video, when you create it to get the media.duration, the media duration value that you will pass to timestampOffset could be wrong, making the glitch even more noticeable, but idk. It will depend on, if media.duration considers the time of the last incomplete segment.

2. Restarting the stream is not easy. you can call record.stop() and then immediately call record.start() but these functions are not synced. This means that the first chunk you get in record.start() is not the continuation of the last chunk received when you called record.stop(). So this means that for each small video that you create you will have a double glitch, first because you drop segments and then because you didn't record anything since the time you called stop till the time you called start.

I am missing something? or this is just not possible? If this is possible please let me know how.

In case this is not possible, then a possible solution is to be able to restart the stream WITHOUT loosing any kind of information.

Trying to explain this as in the specification: 

restart(optional unsigned long timeslice)
1. [....all the irrelevant items of the specification...]
2. Fire a blob event named dataavailable at recorder with blob. The blob MUST contain a complete segment and not a partial segment. The event could just slice the blob to use the last known segment(the remaining data MUST not be dropped) or the event could wait for the remaining data to complete a segment.
3. Restart the recording
4. dispatch an onrestart event
5. In case the blob was sliced, then the data of that blob MUST be used whenever dataavailable is dispatched.

------------------

This stuff is just too technical for me, I have no fear to say nonsense. Could we maybe have a way to be able to feed source_buffer.appendBuffer with data that comes from the middle of the stream and on? Like in saving the metada of the recording and have a way to get the "initialization blobs"  that will allow source_buffer.appendBuffer to work.

```javascript
// streamer.js

var recorder = new MediaRecorder(stream)
recorder.ondataavailable = function(e) {
 if (e.data.size > 0) {
  send_data_to_client_as_arraybuffer(e.data)
 }
}
recorder.getinitializationblobs = function(e) {
 if (e.data.size > 0) {
  return e.data
 }
}
recorder.start(1000)

// client.js

source_buffer.appendBuffer(initializationBlobs)

while(chunks_from_the_middle_of_the_stream_and_on)
source_buffer.appendBuffer(chunks_from_the_middle_of_the_stream_and_on)

```

This will still have the problem that ondataavailable may returns a blob that has the end segment of a previous sent blob. Maybe we should have getinitializationblobs and onsegmentavailable.

This API looks like incomplete, Please inform me, thank you

disclaimer: I just been reading for days about all this and some things I state here could be just my perception and not the reality of whats going on, on which case I would welcome if you can inform me.





Please view or discuss this issue at https://github.com/w3c/mediacapture-record/issues/178 using your GitHub account

Received on Sunday, 11 August 2019 13:10:57 UTC