- From: Thomas Garry via GitHub <sysbot+gh@w3.org>
- Date: Fri, 13 Jan 2023 05:03:48 +0000
- To: public-webrtc-logs@w3.org
I have an application using a cropped track, but I want to be able to hide the video element without causing the feed to be "muted" so that the stream still transmits pixels. To be more clear, the crop target is visible and the video it is transmitting to is not visible. Can this be considered? Currently, https://www.w3.org/TR/mediacapture-region/#empty-crop-target implies that a hidden video element will be an empty crop target and therefore will be muted. To prove this to myself, in my application every time I hide the video element I use with css (i.e. visibility: none or position off the page), the onmute event of the videoTrack from the stream (from navigator.mediaDevices.getDisplayMedia) is fired. I am wondering if showing the cropped video is strictly necessary? Would giving the option to not mute the stream to developers reduce load for users by not requiring the client to have a video element (i.e. not requiring developers to have video.srcObject = stream;)? Currently it seems it's not possible to have a crop target without setting the `srcObject` of a video element and having that video be within user view because the videotrack will automatically be muted when it's not attached to a visible video element. If not just for my specific use case, I believe it may be useful to allow sending the stream to an empty crop target if a developer wanted to use the video track/stream only to send to a web worker that processes it. Example with navigator api, without crop target, without video tag but still streaming: The purpose of the following investigation is to contrast the handling of crop target's mute with other streaming sources. https://github.com/w3c/webcodecs/blob/main/samples/capture-to-file/encode-worker.js ^ this example uses workers to parse video from `new MediaStreamTrackProcessor(videoTrack)`, but also includes `video.srcObject = stream;`. It also uses the `navigator api` to get client video to this processor. However, none of the apis being used in this example restrict developers to having the video element on the window be visible. Since this example is using the navigator api to send to a stream I would expect this to also have a "muted" mechanism for invisible video tags, but it does not. Further checking this example at the live app: https://webcodecs-samples.netlify.app/capture-to-file/capture-to-file.html if you were to inspect element and change the video tag to have the css `visibility: none` or change the positioning `position: absolute; left: 1000000px;` you still get the recording as intended and there is no "mute" functionality even though the video is not present. A developer can even not include the video element at all (go to this website, inspect element, delete the html video tag, click record and verify the recording was still taken)! Even without the video tag, processing of the video was able to happen. TLDR: Another video stream reference I've seen with the navigator API does not require the video tag to be present so does not seem to be using "muted" in a consistent way. Additionally, some apps may want to do video processing with workers and not necessarily want to show the unprocessed video back to the user, so I believe it may possibly be useful to allow streaming to an empty crop target (by not muting the track if the video is not in frame). I apologize if this is the incorrect place to be mentioning all of this, thanks for reading! -- GitHub Notification of comment by thomas-gg Please view or discuss this issue at https://github.com/w3c/mediacapture-region/issues/9#issuecomment-1381319049 using your GitHub account -- Sent via github-notify-ml as configured in https://github.com/w3c/github-notify-ml-config
Received on Friday, 13 January 2023 05:03:50 UTC