Re: Screensharing: Bootstrapping Collaboration between Capturer and Capturee

Great conversation (happening over three different platforms)! Apologies in
advance for butchering the thread with snippet-replies as I catch up.

On Thu, Jun 10, 2021 at 8:21 AM Youenn Fablet <youenn@apple.com> wrote:

> Interesting idea Sergio.
> I wonder whether https://w3c.github.io/mediasession/#actions-model could
> be a source of inspiration here.
> Already defined actions include actions like toggling camera/microphone or
> play/pause. They are triggered by user interacting with UA UI and web
> application registering to those actions.
>
> I wonder whether next slide/previous slide could be defined as actions.
> Capturee would register to those actions.
> Capturer would somehow trigger those actions based on a bootstrap
> mechanism tied to getDisplayMedia.
> Or UA specific UI would allow trigger them in case UA is smart enough to
> understand what is happening.
>

Great idea Sergio and great catch! We may have been focusing too narrowly
here. I like this direction, and see the benefits of standardizing some
basic (opt-in) *presentation** session actions* that could work across all
combinations of products. All parties would see benefits.

On Fri, Jun 11, 2021 at 4:05 AM Youenn Fablet <youenn@apple.com> wrote:

>
> On 11 Jun 2021, at 09:38, Sergio Garcia Murillo <
> sergio.garcia.murillo@gmail.com> wrote:
> I am not representing any VC service or content provider, so I don't have
> a strong opinion about this. But when we analyze this new API we should be
> aware of the consequences and the changes on the "status quo" in the webrtc
> community and acknowledge that some VC services will have competitive
> advantages over others that currently they don't have.
>
> Not sure if that is a good or bad thing, or even if the W3C should be
> worried about this kind of stuff, but let's not be naive and say that all
> VC services will benefit equally from this api.
>
> I think this is absolutely something the W3C should worry about. Silos are
antithetical to standards, and I know at Mozilla we want to encourage a
healthy web ecosystem.

On Thu, Jun 10, 2021 at 4:27 PM Elad Alon <eladalon@google.com> wrote:
>
>> On Thu, Jun 10, 2021 at 3:09 PM Sergio Garcia Murillo <
>> sergio.garcia.murillo@gmail.com> wrote:
>>
>>> So Jitsi will have to implement google docs apis, ms office apis, to do
>>> basically the same? doesn't seem very scalable
>>>
>>
More importantly, *users lose*. Presenters shouldn't have to redo their
presentations based on where they present, something often out of their
control. Users should be able to use their favorite presentation software,
and offer a baseline presentation of it on any VC platform. Adding slide
controls to that baseline seems like a win.


>>    1. I still agree that a more presentation-focused API would also be
>>    useful. Operative word "also." The more general-purpose tool of Capture
>>    Handle has its place. Some collaborations are very specialized by nature,
>>    and scaling them is not necessary. Special-purpose collaborations set up at
>>    great expense between one application (Underwater-Photoshop-2000) and one
>>    VC application (Jitsi) at great expense, financed by one special customer
>>    (the Atlantis School of Underwater Economics).
>>    2. Official and unofficial protocols usually arise in such
>>    situations. The capture-handle exposed by service X could be a JSON which
>>    includes {protocol: "public-preso-2.3", id: "0x382abc42", ...}. That would
>>    make things quite scalable.
>>
>>
I'm glad there's interest in a more presentation-focused API. It might be
useful to separate out the use cases a bit:

   1. The basic slide controls (next, previous, top, bottom?) use case.
   2. Remote browsing, remote collaboration, and other domain-specific
   actions requiring authentication.

I'm not immune to the value of 2 — lots of awesome new work happens first
in silos — but it seems 1 solves the OP use cases better and across more
properties. 1 also seems better aligned with what standards are here to do:
provide interop.

I really like apis when they allow untrusted parties to collaborate
> securely allowing more freedom to the end user and I think it is a goal  it
> is worth pursuing.
>
>
> Communication channels between two untrusted parties is always something
> to look closely at, from a security and privacy standpoint.
> A safe model is to have the UA in the middle: the UA triggers these
> actions on behalf of the user, not on behalf of the capturer.
> For instance, by presenting UA UI in the capturer page to control capturee
> through actions, similar to picture-in-picture for VC.
> For instance if capturee page is playing a video, it might be convenient
> from capturer page to pause the capturee video using the play/pause actions.
> A further step would be to allow capturer page to blend well with this UA
> UI.
>

So I hear 3 directions for presentation control actions:

   - A: UA triggers these standard actions on behalf of the capturer.
   - B: UA triggers these standard actions on behalf of the user, not on
   behalf of the capturer.
   - C: Out of band between mutually participating properties only, based
   on id.

I think both A and B sound promising. Youenn, it seems to me A might have
slightly better ergonomics, so I'm curious about what risks B might
mitigate.

.: Jan-Ivar :.