Re: getThisTabMedia

Very interesting, I'm glad we're discussing use cases.

My focus is very much about ongoing video conferences, so pardon me if I
focus on that one.

The idea of sharing a current tab into an ongoing conference sounds
satisfies the same use as the one I had originally in mind - making it
easier to pick the tab that you want to share. So we'd need to define the
sharing target when on the tab: is it the current or another tab where the
PeerConnection is running?

I also like the idea of cobrowsing through a simple API call, but I don't
think a simple "share this tab" is sufficient for that. I understand by
cobrowsing that everyone who sees the page is able to interact with it and
everyone else sees the same effects of that interaction.

Just some further food for thought.

Cheers,
Silvia.

On Mon, Oct 5, 2020 at 4:29 AM Elad Alon <eladalon@google.com> wrote:

> @Silvia: (1) The proposal is for sharing the tab from which the API call
> is made; no identification is therefore necessary. (2) I agree that one
> wouldn't typically call such an API from the tab where a video-conference
> is active. Rather, one would call it from another tab, where there is
> something which one wishes to share; potentially with a video conference
> (a), but not necessarily (b). An example for (a) would be presenting a
> document from the current tab with a VC happening in another tab. An
> example of (b) would be a video game which allows recording a video of
> itself to a file.
>
> @Sergio: Sharing only part of a tab is also something I am looking into,
> so you *might* see a proposal to that effect come from me soon. I see the
> connection between the two issues (share-this-tab and region-capture);
> namely, one would likely want to limit a capture to a certain portion of a
> tab, only if one knows the contents of that tab, and that'd typically only
> be true for the current-tab. But I don't think it's necessary to tackle
> region-capture before reaching consensus on capture-this-tab, since
> capture-this-tab is useful even without region-capture. Or wdyt?
>
> On Sun, Oct 4, 2020 at 1:57 PM Sergio Garcia Murillo <
> sergio.garcia.murillo@gmail.com> wrote:
>
>> There is life outside of the videoconferencing world.. :)
>>
>> This API could be used to replace cobrowsing, when a user starts a PC to
>> share the contents of the current page to get support from a remote agent
>> for example. It would be also interesting to be able to create a OBS like
>> broadcasting editor, when you can switch/mix different sources to create a
>> video feed. Last, but not least, for videoconferences, you could use it for
>> rebroadcast current conference to a webrtc-cdn for reaching massive scale
>> (not that I have an immediate use for this ;)
>>
>> While I find this interesting, as it will eliminate the potential error
>> of choosing the wrong tab by having to manually pick it (especially
>> interesting for the remote use case), I am wondering if we could enhance it
>> by allowing to specify concrete DOM elements to be capture or regions of
>> the visible window.
>>
>> For example on the OBS-like video editor, you could choose to only share
>> the region of the resulting  composition window, but not the controls.
>>
>> [image: upload.wikimedia.org/wikipedia/commons/b/b9/OBS...]
>>
>> Also, it could be interesting to be able to capture at a different
>> resolution than the displayed one. For example in the previous example, it
>> would be interesting to be able to capture a 4K stream from the composition
>> window without having to actually display a 4K element (i.e. you could
>> display the element with a zoom style of 25%, but capture full size).
>>
>> Best regards
>> Sergio
>>
>> On 04/10/2020 13:09, Silvia Pfeiffer wrote:
>>
>> You wouldn't share the current tab - the current tab is the one where the
>> video call is taking place. You'd typically want to share one of the other
>> open tabs as a screenshare. That's how I understand it at least.
>>
>> On Sun, Oct 4, 2020 at 9:58 PM Dan Jenkins <dan@nimblea.pe> wrote:
>>
>>> Isn't it just the current tab - there is no selecting done by the user -
>>> a better name would likely be getCurrentViewportMedia or something along
>>> those lines
>>>
>>> On Sun, 4 Oct 2020, 02:44 Silvia Pfeiffer, <silviapfeiffer1@gmail.com>
>>> wrote:
>>>
>>>> I love the idea.
>>>>
>>>> How do you suggest the application would be able to identify the "tab"
>>>> without the user selecting it?
>>>> Would it be based on a text filter, e.g. getTabDisplayMedia("Screen
>>>> Capture") would then identify the "Screen Capture" tab because of its
>>>> naming?
>>>> Would it automatically pick the first such available or would it bring
>>>> up a picker dialog again if there are several matching ones?
>>>>
>>>> Cheers,
>>>> Silvia.
>>>>
>>>>
>>>> On Sat, Oct 3, 2020 at 12:40 AM Elad Alon <eladalon@google.com> wrote:
>>>>
>>>>> Hello all,
>>>>>
>>>>> We're proposing an API that would be very similar to
>>>>> navigator.mediaDevices.getDisplayMedia
>>>>> <https://www.w3.org/TR/screen-capture/#dom-mediadevices-getdisplaymedia>,
>>>>> but which only will be able to initiate capturing of the tab from which it
>>>>> is called (rather than provide the much wider selection of share-sources
>>>>> that getDisplayMedia has).
>>>>>
>>>>> In the case of getDisplayMedia, the Screen Capture TR
>>>>> <https://www.w3.org/TR/screen-capture/> says (bold-emphasis mine):
>>>>>
>>>>>> The user agent MUST let the end-user choose which display surface to
>>>>>> share *out of all available choices* every time, and MUST NOT use
>>>>>> constraints to limit that choice.
>>>>>
>>>>>
>>>>> This will be true for getThisTabMedia too, but there, the only
>>>>> “available choice” is the surface from which it is called (specifically,
>>>>> the browser tab). We’ll be sending a PR similar to this one
>>>>> <https://github.com/eladalon1983/mediacapture-screen-share/pull/3>
>>>>> against the Screen Capture TR fairly soon.
>>>>>
>>>>> The *main benefit* of this new API is that a simple accept/reject
>>>>> dialog can be presented to the user, rather than a complex picker
>>>>> (screen/window/tab).
>>>>>
>>>>> *Contrast getDisplayMedia picker in Chrome:*
>>>>> [image: image.png]
>>>>> *Potential dialog-box for getThisTabMedia:*
>>>>> [image: image.png]
>>>>>
>>>>> We think that the security properties of own-tab capture are no worse
>>>>> than the version that goes via the picker. We note that the application
>>>>> will have control over the surface that is being displayed, and that can
>>>>> cause some sharing of information that would otherwise be inaccessible,
>>>>> such as colors on visited links, or content of embedded frames, but we
>>>>> think that the risks are no bigger than for regular sharing, and that the
>>>>> proposed, simple, prompt is good enough to mitigate this concern.
>>>>>
>>>>> Your feedback is hereby solicited. :-)
>>>>>
>>>>> Thanks,
>>>>> Elad Alon
>>>>>
>>>>
>>