Re: getThisTabMedia

@Silvia: Our proposal is to model
getThisTabMedia/getCurrentViewportMedia/getOtherName after getDisplayMedia.
The result of getDisplayMedia is a (promise-for-a-)MediaStream. My
understanding is that it is up to the application, once it gets hold of
that MediaStream, to determine what to do with that MediaStream; possibly,
as an example for your use-case, transmitting the media to the same SFU as
that of the VC happening in the other tab; this transmission could use a
dedicated connection in the current tab.

@Dan
1. Sorry for not acknowledging your earlier email. I have no strong feeling
for either name, and would gladly run with any; I leave this in others'
most capable hands.
2. Could you please point me to the GitHub issue to which you're referring?
As mentioned, I *might* (no promise) be proposing something relating to
region-capture soon, and if so, I'd love to read your thoughts beforehand.
3. About region-capture - this is an interesting topic that produces its
own set of challenges. To name just one that I think is of particular
interest - window resizing. When this happens, it is imperative to pause
the capture until the application can describe the newly desired capture
area, lest out-of-region content be momentarily captured and possibly
transmitted. I think this topic encompasses enough challenges, that it
would be best to handle it as a follow-up; I think get-this-tab unlocks
some nice possibilities for users on its own.




On Mon, Oct 5, 2020 at 8:56 AM Dan Jenkins <dan@nimblea.pe> wrote:

> Did my reply to Silvia get posted in the end other than in her email
> history? (Long time lurker... New poster)
>
> Everything's been covered now other than my proposed API name change to
> something along the lines of getCurrentViewportMedia but that doesn't work
> if you're going to allow selecting a specific area to share, maybe?
>
> Elad, I recently brought up the need for being able to share a portion of
> a window/display/tab using getDisplayMedia in a GitHub issue - the spec
> says that the stream can be a portion of a display... and in my example (I
> have 2 4k monitors, sharing a whole display doesn't make sense) it makes
> sense for the browser to do the selecting of the area for user
> confidence/implementation into every web app using getDisplayMedia...
> Something to think about with what you're thinking for selecting specific
> areas/elements to share etc....
>
> On Sun, 4 Oct 2020, 18:29 Elad Alon, <eladalon@google.com> wrote:
>
>> @Silvia: (1) The proposal is for sharing the tab from which the API call
>> is made; no identification is therefore necessary. (2) I agree that one
>> wouldn't typically call such an API from the tab where a video-conference
>> is active. Rather, one would call it from another tab, where there is
>> something which one wishes to share; potentially with a video conference
>> (a), but not necessarily (b). An example for (a) would be presenting a
>> document from the current tab with a VC happening in another tab. An
>> example of (b) would be a video game which allows recording a video of
>> itself to a file.
>>
>> @Sergio: Sharing only part of a tab is also something I am looking into,
>> so you *might* see a proposal to that effect come from me soon. I see
>> the connection between the two issues (share-this-tab and region-capture);
>> namely, one would likely want to limit a capture to a certain portion of a
>> tab, only if one knows the contents of that tab, and that'd typically only
>> be true for the current-tab. But I don't think it's necessary to tackle
>> region-capture before reaching consensus on capture-this-tab, since
>> capture-this-tab is useful even without region-capture. Or wdyt?
>>
>> On Sun, Oct 4, 2020 at 1:57 PM Sergio Garcia Murillo <
>> sergio.garcia.murillo@gmail.com> wrote:
>>
>>> There is life outside of the videoconferencing world.. :)
>>>
>>> This API could be used to replace cobrowsing, when a user starts a PC to
>>> share the contents of the current page to get support from a remote agent
>>> for example. It would be also interesting to be able to create a OBS like
>>> broadcasting editor, when you can switch/mix different sources to create a
>>> video feed. Last, but not least, for videoconferences, you could use it for
>>> rebroadcast current conference to a webrtc-cdn for reaching massive scale
>>> (not that I have an immediate use for this ;)
>>>
>>> While I find this interesting, as it will eliminate the potential error
>>> of choosing the wrong tab by having to manually pick it (especially
>>> interesting for the remote use case), I am wondering if we could enhance it
>>> by allowing to specify concrete DOM elements to be capture or regions of
>>> the visible window.
>>>
>>> For example on the OBS-like video editor, you could choose to only share
>>> the region of the resulting  composition window, but not the controls.
>>>
>>> [image: upload.wikimedia.org/wikipedia/commons/b/b9/OBS...]
>>>
>>> Also, it could be interesting to be able to capture at a different
>>> resolution than the displayed one. For example in the previous example, it
>>> would be interesting to be able to capture a 4K stream from the composition
>>> window without having to actually display a 4K element (i.e. you could
>>> display the element with a zoom style of 25%, but capture full size).
>>>
>>> Best regards
>>> Sergio
>>>
>>> On 04/10/2020 13:09, Silvia Pfeiffer wrote:
>>>
>>> You wouldn't share the current tab - the current tab is the one where
>>> the video call is taking place. You'd typically want to share one of the
>>> other open tabs as a screenshare. That's how I understand it at least.
>>>
>>> On Sun, Oct 4, 2020 at 9:58 PM Dan Jenkins <dan@nimblea.pe> wrote:
>>>
>>>> Isn't it just the current tab - there is no selecting done by the user
>>>> - a better name would likely be getCurrentViewportMedia or something along
>>>> those lines
>>>>
>>>> On Sun, 4 Oct 2020, 02:44 Silvia Pfeiffer, <silviapfeiffer1@gmail.com>
>>>> wrote:
>>>>
>>>>> I love the idea.
>>>>>
>>>>> How do you suggest the application would be able to identify the "tab"
>>>>> without the user selecting it?
>>>>> Would it be based on a text filter, e.g. getTabDisplayMedia("Screen
>>>>> Capture") would then identify the "Screen Capture" tab because of its
>>>>> naming?
>>>>> Would it automatically pick the first such available or would it bring
>>>>> up a picker dialog again if there are several matching ones?
>>>>>
>>>>> Cheers,
>>>>> Silvia.
>>>>>
>>>>>
>>>>> On Sat, Oct 3, 2020 at 12:40 AM Elad Alon <eladalon@google.com> wrote:
>>>>>
>>>>>> Hello all,
>>>>>>
>>>>>> We're proposing an API that would be very similar to
>>>>>> navigator.mediaDevices.getDisplayMedia
>>>>>> <https://www.w3.org/TR/screen-capture/#dom-mediadevices-getdisplaymedia>,
>>>>>> but which only will be able to initiate capturing of the tab from which it
>>>>>> is called (rather than provide the much wider selection of share-sources
>>>>>> that getDisplayMedia has).
>>>>>>
>>>>>> In the case of getDisplayMedia, the Screen Capture TR
>>>>>> <https://www.w3.org/TR/screen-capture/> says (bold-emphasis mine):
>>>>>>
>>>>>>> The user agent MUST let the end-user choose which display surface to
>>>>>>> share *out of all available choices* every time, and MUST NOT use
>>>>>>> constraints to limit that choice.
>>>>>>
>>>>>>
>>>>>> This will be true for getThisTabMedia too, but there, the only
>>>>>> “available choice” is the surface from which it is called (specifically,
>>>>>> the browser tab). We’ll be sending a PR similar to this one
>>>>>> <https://github.com/eladalon1983/mediacapture-screen-share/pull/3>
>>>>>> against the Screen Capture TR fairly soon.
>>>>>>
>>>>>> The *main benefit* of this new API is that a simple accept/reject
>>>>>> dialog can be presented to the user, rather than a complex picker
>>>>>> (screen/window/tab).
>>>>>>
>>>>>> *Contrast getDisplayMedia picker in Chrome:*
>>>>>> [image: image.png]
>>>>>> *Potential dialog-box for getThisTabMedia:*
>>>>>> [image: image.png]
>>>>>>
>>>>>> We think that the security properties of own-tab capture are no worse
>>>>>> than the version that goes via the picker. We note that the application
>>>>>> will have control over the surface that is being displayed, and that can
>>>>>> cause some sharing of information that would otherwise be inaccessible,
>>>>>> such as colors on visited links, or content of embedded frames, but we
>>>>>> think that the risks are no bigger than for regular sharing, and that the
>>>>>> proposed, simple, prompt is good enough to mitigate this concern.
>>>>>>
>>>>>> Your feedback is hereby solicited. :-)
>>>>>>
>>>>>> Thanks,
>>>>>> Elad Alon
>>>>>>
>>>>>
>>>