W3C home > Mailing lists > Public > public-webrtc-logs@w3.org > April 2019

Re: [mediacapture-screen-share] Provide a means to select only part of a screen to capture (#105)

From: guest271314 via GitHub <sysbot+gh@w3.org>
Date: Mon, 29 Apr 2019 21:32:47 +0000
To: public-webrtc-logs@w3.org
Message-ID: <issue_comment.created-487752095-1556573566-sysbot+gh@w3.org>
@youennf Am only interested in concatenating multiple media files into a single media file. While attempting to achieve that requirement have tried several approaches where summaries of some of the approaches tried can be found at each branch of the above-linked repository.

> One is for a browser to allow a user selecting a part of a screen. I do not see benefits in adding a constraint for that so this seems like an implementation decision, not a spec one.
> Implementing this partial screen selection probably need some thoughts. For instance, how to present to users which part of the screen is being captured.

The live `window`, `tab`,  `appliacation` is _already_ presented to the user at Chromium 73, minimized in a grid display. Firefox provides a dropdown next to the location bar. The selected screen at the grid at Chromium can be scaled to 2/3 of the screen _if_ the user selects a radio to toggle on _exact_ selection, in similar manner that Firefox Developer Tools provides a means to select _part_ of a screen. Notice, the specification already has an `exact` constraint modifier. This proposal is not asking for what should already be provided, given a specifier exists named `exact`. The requirement is to capture the _exact_ dimensions of a given display, monitor, screen, application, `window`, `<video>`, etc.

![firefox_developer_tools_take_a_screenshot](https://user-images.githubusercontent.com/4174848/56927695-5a24b480-6ac4-11e9-84b5-9255846ee556.png)



The selected screen can be translated to `sx`, `sy`, `sWidth`, `sHeight` constraints. Alternatively, the user can programmatically set the input constraints directly `videoTrack.applyConstraints({screenX:100, screenY:100, screenWidth, screenHeight})`.

![sx_sy_sWidth_sHeight](https://user-images.githubusercontent.com/4174848/56927607-1fbb1780-6ac4-11e9-9ad2-7e42ee4349ca.png)

Not sure how you can state that you do not see benefits in adding constraints for parts of the screen. If that is the case the specification might as well deprecate `width`, `height`, `resizeMode`, etc. Such an specification extension is consistent with the current specification that lacks such constraints. For the `getDisplayMedia()` API (potentially for `ImageCapture` API), since the entire screen is being captured, it is reasonable to have constraints which select only part of a screen, similar to `Canvas​Rendering​Context2D.draw​Image()` https://developer.mozilla.org/en-US/docs/Web/API/CanvasRenderingContext2D/drawImage implementation 

> `sx` Optional
> The x-axis coordinate of the top left corner of the sub-rectangle of the source image to draw into the destination context.
> 
> `sy` Optional
> The y-axis coordinate of the top left corner of the sub-rectangle of the source image to draw into the destination context.
> 
> `sWidth` Optional
> The width of the sub-rectangle of the source image to draw into the destination context. If not specified, the entire rectangle from the coordinates specified by sx and sy to the bottom-right corner of the image is used.
> 
> `sHeight` Optional
> The height of the sub-rectangle of the source image to draw into the destination context.

-

> 
The second requirement is to capture a browser tab w/o title, location bar, slider... This seems somehow more closely related to cursor. Are you only interested in the latter?

The constraint `cursor:"never"` does not work. Filed an issue at wpt to address the fact that there are no wpt tests coded to independently verify that fact. In the mean time, created workarounds for the non-working constraint using CSS. 

Created a workaround for not recording the local and title bars at Chromium 73 after a day trying different approaches. The original branch of using `getDisplayMedia()` to record multiple media resources used `requestFullScreen()` to avoid having to address not capturing portions of default browser GUI. Tried `kiosk` mode at Chromium, though functionality in that mode is minimal. Decided to revisit attempting to exclude portions of the screen that were not needed to be captured. The above code, modified since posting the comment, does not record the title and location bars at Chromium 73, mainly using CSS. Will dive in to Firefox next. These workarounds can be omitted if existing technologies were incorporated at least into this API, which by its very name is relevant to capturing an entire screen, thus it is reasonable to conclude that there might be portions of an entire screen which would need to be excluded, or, conversely, included to the exclusion of certain parts of the screen. Am only requesting rectangular selection, not triangles or circles, or parallelograms, though those dimensions, too, should ultimately be possible.



-- 
GitHub Notification of comment by guest271314
Please view or discuss this issue at https://github.com/w3c/mediacapture-screen-share/issues/105#issuecomment-487752095 using your GitHub account
Received on Monday, 29 April 2019 21:32:49 UTC

This archive was generated by hypermail 2.3.1 : Tuesday, 4 June 2019 15:32:56 UTC