- From: guest271314 via GitHub <sysbot+gh@w3.org>
- Date: Wed, 21 Aug 2019 20:43:51 +0000
- To: public-webrtc@w3.org
guest271314 has just created a new issue for https://github.com/w3c/mediacapture-main:
== Why does unmute event take between 30 to 60 seconds to dispatch? Does a canonical procedure exist to estimate when unmute will be dispatched? ==
**Background**
At Chromium 78 neither `VP8` nor `VP9` codecs passed to `mimeType` of `MediaRecorder` result in the correct `width` and `height` of the underlying `MediaStreamTrack` are recorded and encoded into the resulting media file https://bugs.chromium.org/p/chromium/issues/detail?id=972470, https://bugs.chromium.org/p/chromium/issues/detail?id=983777. Chromium also has a bug where the `MediaRecorder` outputs a `Blob` with `size` set to `0` when the `MediaStream` is not set as `srcObject` of a `<video>` element https://bugs.chromium.org/p/chromium/issues/detail?id=952700, which could contribute to the issue of `MediaRecorder` not recording the correct `width` and `height`, as the pixel dimensions of the underlying encoded frames appear to be related to the `width` and `height` of the HTML `<video>` element where `srcObject` is set to the `MediaStream` to be recorded.
Chromium supports `avc1` and `h264` codecs at `MediaRecorder` which does record the correct `width` and `height` of the `MediaStreamTrack`, though the code necessary to achieve that requirement is not obvious, and would probably not be reached without a fair amount of testing multiple configurations of code not directly under the Media Streams and Capture specification
Steps to reproduce
1) Set mimeType of MediaRecorder to video/x-matroska;codecs=avc1 or video/x-matroska;codecs=h264; neither VP8 nor VP9 record correct width and height for all each replaced MediaStreamTrack
2) Attach canplay event to <video> element where the MediaStreamTrack of kind video is captured from
3) Within canplay event attach resize event to <video> element where MediaStream to be recorded is set as srcObject
4) Within resize event check if <video> element where the MediaStreamTrack of kind video is captured from width and height are equal to width and height values from MediaStreamTrack of kind video width and height
5) If width and height of <video> where the MediaStreamTrack of kind video is captured from width and height are equal to width and height values from MediaStreamTrack of kind video width and height attach unmute event to the original MediaStreamTrack, not the remote MediaStreamTrack after track event of RTCPeerConnection is dispatched
6) Await unmute event of initial MediaStreamTrack of kind video, approximately 30 seconds
7) When unmute event is dispatched to initial video MediaStreamTrack pass the MediaStream from track even of RTCPeerConnection to MediaRecorder, execute play() on <video> element that where the MediaStreamTrack will be replaced using RTCRtpSender.replaceTrack(), execute start() method of MediaRecorder
Code in pertinent part
```
const videoStream = document.createElement("video");
videoStream.onloadedmetadata = e => {
const {
videoWidth: width,
videoHeight: height
} = e.target;
videoStream.width = video.width = width;
videoStream.height = video.height = height;
}
const ms = videoStream.captureStream();
ms.onaddtrack = async e => {
console.log(e.track.getSettings());
if (e.track.kind === "audio") {
await audioSender.replaceTrack(e.track);
} else {
const {
width, height
} = e.track.getSettings();
await videoSender.replaceTrack(e.track);
}
}
for (const blobURL of urls) {
await new Promise(resolve => {
videoStream.addEventListener("pause", async _ => {
await audioSender.replaceTrack(audioTrack);
await videoSender.replaceTrack(videoTrack);
resolve();
}, {
once: true
});
videoStream.addEventListener("canplay", async _ => {
if (!recorder) {
await new Promise(resizedPromise => {
const handleResize = async _ => {
const {
width, height
} = ms.getVideoTracks()[0].getSettings();
console.log(video.width, width);
if (video.width === width && video.height === height) {
console.log(video.width, width);
video.removeEventListener("resize", handleResize);
await new Promise(unmutePromise => {
const [track] = mediaStream.getVideoTracks();
let now = performance.now();
console.log(video.currentTime);
track.addEventListener("unmute", async _ => {
console.log(_.type, (performance.now() - now) / 1000, video.currentTime);
unmutePromise(await videoStream.play());
}, {
once: true
})
});
recorder = new MediaRecorder(stream, {
mimeType: "video/x-matroska;codecs=h264"
});
recorder.onstart = _ => {
console.log(_.type);
}
result = new Promise(resolve => {
recorder.addEventListener("dataavailable", ({
data
}) => {
resolve(data);
}, {
once: true
})
});
recorder.start();
resizedPromise();
}
}
video.addEventListener("resize", handleResize);
});
} else {
await videoStream.play();
}
}, {
once: true
});
videoStream.src = blobURL;
});
}
recorder.stop();
```
plnkr reproducing the issue https://plnkr.co/edit/Axkb8s.
The portion of the code where the developer would need to simply wait for an unknown output, if any, is **6)**, which is the topic of this issue.
Both of the `RTCPeerConnection` instances are "local". No server is involved.
AFAICT `getStats()` does not provide a value for future events, in this case, `unmute`.
When attempting to record a possibly "infinite" `MediaStream` where the `MediaStreamTrack` of `kind` `"video"` potentially has variable pixel dimensions (`width` and `height`) due to the initial track being replaced using `RTCRtpSender.replaceTrack()` `unmute` event of the initial `MediaStreamTrack` (`CanvasCaptureMediaStreamTrack`) takes between 30 and 60 seconds for `unmute` event to be dispatched. Why?
Does a canonical procedure exist to estimate when `unmute` event attahced to a `MediaStreamTrack` will be dispatched?
Please view or discuss this issue at https://github.com/w3c/mediacapture-main/issues/614 using your GitHub account
Received on Wednesday, 21 August 2019 20:43:53 UTC