Re: Sense of the WG on mediacapture-transform API

Le 18/10/2021 à 19:58, Bernard Aboba a écrit :
> At the WebRTC WG Virtual Interim meeting on Thursday, October 14, 2021
> we discussed the path forward on mediacapture-transform API.  The slides
> from the meeting are available here:
> https://docs.google.com/presentation/d/1rdaRvl3h3viqvndyJVzwIAPtOpMjx_2u7q7MBHUwbyY/
> <https://docs.google.com/presentation/d/1rdaRvl3h3viqvndyJVzwIAPtOpMjx_2u7q7MBHUwbyY/>
> 
> We would now like to get a "sense of the WG" on the following questions:
> 
> Question 1:  Should mediacapture-transform be based on streams? 

Yes - I think we should use streams, unless and until they're proven to
create more problems than they solve.

> Question 2: Should mediacapture-transform support audio? 

I don't think it should be a priority to make it support audio given
that audio processing is already possible. I certainly don't object to
making it work with audio if it comes with only minimal additional work.

> Question 3: Should mediacapture-transform be exposed on the main thread? 

No - I've been hesitating on this question for a while, but I think the
priority of constituencies and the risk that apps get deployed that will
work badly (or not at all) on low-end devices points toward NOT making
it exposed on the main thread.

I can easily believe that developers don't want to have to
rearchitecture their code to use workers - I only rarely use them in
most of (my pretty frequent) JS coding today, and their developer
experience remains unpleasant. I can also see that there are cases where
it doesn't bring direct advantages to go through workers; but by making
it available on the main thread, my assessment is that too many
developers will use it there assuming it'll work fine (since it will
inevitably work OK in their likely high-end development environment),
only to create real issues with end-users with lower end devices. There
are enough situations where the said users don't get to pick a different
service that I don't think we can assume this will be solved just by
competition among service providers.

I could imagine making it available on the main thread with some more
friction for developers (so that it would not be the default path);
maybe a more detailed view of the use cases where having it in the main
thread is the preferable path would also help reduce the risks of having
it too easily mis-used on the main thread.

If we were to proceed with it on the main thread, I would want the spec
to discourage its use there.

All in all, I think we should at the very least start with it NOT
available on the main thread, and only add it there if we can convince
ourselves the risks can be mitigated enough.

(This question reminds me of the debates around synchronous XHR - which
was certainly convenient and relied on by quite a few developers, and
all the more so that the asynchronous primitives in JS were pretty poor
back then (a bit like the poor developer experience with workers today);
but sync XHR ended up getting deprecated, with big migration costs to
users, developers and browser makers)

Dom

Received on Monday, 25 October 2021 13:20:59 UTC