Re: [mediacapture-screen-share] Add ability to crop a MediaStream obtained through the share-this-tab API (#158)

> Why, are you planning to blur the desktop? Or add overlays? Serious question, because, if you are, such app logic may belong more naturally in the top-level page

I foresee that video-conferencing applications will be using sophisticated upcoming WebRTC features, and some of these do not require any knowledge of the application or the video-content being captured from it. One example is the intention to allow applications to inject certain WebRTC modules. Another example is Web Codecs.

> A complexity sufficient to scare away web developers from using browser features

I argue that WebRTC is evolving to the point where power users - like applications **centered** around video-conferencing - can do much, but that this comes with added complexity when compared to less sophisticated WebRTC usage. Note that this complexity does **not** arise from the cropping-feature which I propose. The complexity I am referring to comes from the addition of Web Codecs, Breakout Box, module-injection, etc. - all of which are orthogonal to the proposed cropping feature. A non-conferencing application like Slides might wish to enjoy the benefits of all of these sophisticated features, despite not itself being a sophisticated WebRTC user.

True, enjoying power-features can be achieved through embedding a library. It can also be achieved by embedding a specialized application in an iframe. I don't think we should restrict applications to only one of these without good reason. IMHO, we don't have good reason here, as supporting both use cases (library/iframe) is easy.

> > I don't think large applications like Slides/Docs, that normally do not handle any video-conferencing, should be expected to take on that responsibility.
> 
> I'd like to push back on this. I actually think they do.

I'm in contact with Google Slides, and I know that they would prefer to keep conferencing logic in the Meet iframe. I think that browsers need to take into account the preferences of applications, and IMHO Google Slides' case is likely to be common to other applications as well. That is, the case of a large non-conferencing application wishing to embed a large conferencing-specific application in an iframe. Please note how little work would be needed to integrate these two applications if we use my twin suggestions of share-this-tab and crop-to-div(-by-id). Integration could then be accomplished with merely two messages posted between these frames - the video-conferencing iframe posts "I wish to start capturing content," and non-conferencing top-level application responds "the DIV you're looking for can be referenced through the crop-ID 'ca28ff965d'."

> Last, but not least, for this feature to be safe, it needs to be secured by something like COEP, as I propose in #155. Those restrictions would need to apply to the top-level page, so we might as well assume it will need to be deeply involved.

This is an orthogonal issue where we'll hopefully reach orthogonal agreement. I think that in the current discussion, it may safely be assumed that share-this-tab is indeed gated by your original COEP-based suggestion. (I mean, even if it ends up not being the case, we can discuss the interaction of cropping with whatever other gating we end up using once we know what that is.)

-- 
GitHub Notification of comment by eladalon1983
Please view or discuss this issue at https://github.com/w3c/mediacapture-screen-share/issues/158#issuecomment-769439548 using your GitHub account


-- 
Sent via github-notify-ml as configured in https://github.com/w3c/github-notify-ml-config

Received on Thursday, 28 January 2021 22:24:46 UTC