Re: [mediacapture-screen-share] Multi-capture (concurrent capture of multiple surfaces) (#204)

@eladalon1983 yes z-order is useful
(currently, the demo first paints all selected window at their last seen position, and then paints over them the recently found windows, so z-order works, but of course it relies on image processing, which we want to avoid completely)

The native versions of Screegle for Mac and Windows poll the OS for window information.

On MacOS in Swift or ObjectiveC, the function is `CGWindowListCopyWindowInfo` which returns the list of all windows ordered from back to front, and contains the `kCGWindowBounds` rectangle of each window (among other things).

(on Windows, Screegle uses ElectronJS and relies on https://github.com/sentialx/node-window-manager/blob/master/src/classes/window.ts#L22 to obtain window information, matching the windowID to the DesktopCapturer.getSources https://www.electronjs.org/docs/latest/api/desktop-capturer which conveniently uses `window:native_window_ID` as a way to represent available windows; which is what `getCapabilities` returns in `deviceId` for a given stream's track.

Perhaps the easiest way would be to extend the existing `getCapabilities` https://www.w3.org/TR/mediacapture-streams/#dictionary-mediatrackcapabilities-members API by adding `{top,left,zOrder}`? (`width` and `height` are already returned)

Or allow a web application that could hold several streamIDs a call on the 
`navigator.mediaDevices.streamInfo(<array of streamIDs>)` and return an array of information about these streams orders from back to front, for example `[{streamID1,x1,y1,w1,h1},...,{streamID4,x4,y4,w4,h4}]` (here the window corresponding to streamID4 is thus above all other windows listed in the call)

The web application would be able to call this API at any time.


-- 
GitHub Notification of comment by ldenoue
Please view or discuss this issue at https://github.com/w3c/mediacapture-screen-share/issues/204#issuecomment-1038947690 using your GitHub account


-- 
Sent via github-notify-ml as configured in https://github.com/w3c/github-notify-ml-config

Received on Monday, 14 February 2022 11:02:56 UTC