- From: Dominique Hazael-Massieux <dom@w3.org>
- Date: Wed, 13 Dec 2023 08:50:04 +0100
- To: public-webrtc@w3.org
Hi,
The minutes of our WG meeting yesterday are available at:
https://www.w3.org/2023/12/12-webrtc-minutes.html
and copied as text below.
Dom
WebRTC December 12 2023 meeting
12 December 2023
[2]Agenda. [3]IRC log.
[2] https://www.w3.org/2011/04/webrtc/wiki/December_12_2023
[3] https://www.w3.org/2023/12/12-webrtc-irc
Attendees
Present
Bernard, Dom, Elad, Fippo, Guido, Harald, Jan-Ivar,
PatrickRockhill, PeterThatcher, TimPanton, TonyHerre,
Tove, Varun, Youenn
Regrets
-
Chair
Bernard, HTA, Jan-Ivar
Scribe
dom
Contents
1. [4]Screen Capture
1. [5]Issue #276: Handling of contradictory hints
2. [6]Issue #281: Distinguish cancellations from absent
OS permissions
2. [7]Solve user agent camera/microphone double-mute
(mediacapture-extensions)
3. [8]Dynamic Switching in Captured Surfaces
4. [9]RtpSender Encoded Source
5. [10]Keyframe API
6. [11]Summary of resolutions
Meeting minutes
Slideset: [12]https://lists.w3.org/Archives/Public/www-archive/
2023Dec/att-0002/WEBRTCWG-2023-12-12__1_.pdf
[12]
https://lists.w3.org/Archives/Public/www-archive/2023Dec/att-0002/WEBRTCWG-2023-12-12__1_.pdf
[13]Screen Capture
[13] https://github.com/w3c/mediacapture-screen-share
Issue [14]#276: Handling of contradictory hints
[14] https://github.com/w3c/mediacapture-screen-share/issues/276
[15][Slide 11]
[15]
https://lists.w3.org/Archives/Public/www-archive/2023Dec/att-0002/WEBRTCWG-2023-12-12__1_.pdf#page=11
[16][Slide 12]
[16]
https://lists.w3.org/Archives/Public/www-archive/2023Dec/att-0002/WEBRTCWG-2023-12-12__1_.pdf#page=12
[17][Slide 13]
[17]
https://lists.w3.org/Archives/Public/www-archive/2023Dec/att-0002/WEBRTCWG-2023-12-12__1_.pdf#page=13
[18][Slide 14]
[18]
https://lists.w3.org/Archives/Public/www-archive/2023Dec/att-0002/WEBRTCWG-2023-12-12__1_.pdf#page=14
[+1s from fippo, bernard, harald]
Jan-Ivar: this shows maybe we went a bit fast with these hints;
the UA is free not to react to them
… I would just ignore them
Bernard: I've noticed this pattern with hints in other places
… it's problematic - this feels like bad config
… maybe they should not be hints
elad: certain constraints can be self-contradictory leading to
overconstrained error
Timp: what's the goal here? notify the developer they set an
incompatible set of options? protecting the user from something
bad?
Elad: what error should be thrown interoperably?
Timp: but what would happen with the error thrown?
Elad: the developer should fix their code
TimP: you want it to fail then?
… this is probably the right thing to do
Elad: having fail it the same across browsers would be good
Harald: +1 for specifying something; I don't it matters so much
what is specified - ignoring may be OK in some cases
Elad: +1 to focus on interop first and foremost
bernard: some tricky aspects: given the goal is to guide
developers, how would they be guided toward their mistake?
… if you ignore or reject, it needs to be clear to the
developer what was ignored/rejected, and what remains in the
end
Elad: the error could point to the list of things
allowed/disallowed
… ignoring the hints makes it probably trickier to avoid
unexpected results
Bernard: can you retrieve what was eventually applied?
Elad: if you reject, nothing is applied
Bernard: but what if you ignore?
Elad: hence why I think rejecting is the right thing
Youenn: normally we try to minimize API to avoid contradictory
choices - we should avoid that moving forward
… here, rejecting to get interop is OK
Jan-Ivar: we have to take into account display switching (as we
will discuss)
… looking at a PR will help
Elad: the PR for display switching has some discussion on how
it is impacted by constraints
RESOLUTION: Consensus to specify an interoperable behavior and
iterate initially on a pull request to be proposed by Elad
Issue [19]#281: Distinguish cancellations from absent OS permissions
[19] https://github.com/w3c/mediacapture-screen-share/issues/281
[20][Slide 15]
[20]
https://lists.w3.org/Archives/Public/www-archive/2023Dec/att-0002/WEBRTCWG-2023-12-12__1_.pdf#page=15
[21][Slide 16]
[21]
https://lists.w3.org/Archives/Public/www-archive/2023Dec/att-0002/WEBRTCWG-2023-12-12__1_.pdf#page=16
Elad: Jan-Ivar mentioned that permission.query might serve this
purpose; I wonder if it introduces a fingerprinting concern
Jan-Ivar: you're right that it would, but the mid-term solution
proposed in the issue would match mitigation for
permission.query, so I think we could make that work
Elad: would that work with Safari in embargo mode?
… when the user cancels 3 times a call
Youenn: this is under the control of the UA; it could report
"blocked" under these circumstances
Elad: but would there be a way to distinguish OS-blocked vs
user-blocked?
Harald: if we want to have distinguishable situations, there
needs to be matching API surface
Youenn: I'm not sure embargo-mode or not is relevant
… what Elad is asking for is to tell the user that "something
happened"
… if we have more than OS-rejected vs user-reject, an enum is
needed
… otherwise, the permission API may be enough
… it would still need to be specified given that
permission.query is still a bit fuzzy
… do you foresee more values in the enum?
Elad: another problematic scenario
… an app being relaunched may not be able to determine if it's
blocked because of the user vs OS without calling
getDisplayMedia
Youenn: that's true of your proposal as well
… so for the enum, do you see more values?
Elad: a boolean may be OK but harder to extend
… the new API I propose would work better e.g. if a browser in
the future decides to embargo permission after a single call
Jan-Ivar: I would object going further to permission.query - we
shouldn't reveal an OS level decision, the user agent is the
party
Elad: how about boolean "user-rejected" yes or no?
Jan-Ivar: that's equivalent to permission.query
Elad: the user won't know what they can do to solve the
situation
Jan-Ivar: that's up to the UA to guide them
Elad: I think we should empower the app to help the user
instead of always relying on the UA
Youenn: this isn't specific to getDisplayMedia - this would
probably apply to mic and cameras
… how are we dealing with this?
… if it's a problem worth solving, it should be solved across
sources
Elad: this solution could be extended to getUserMedia
Youenn: I think we should explore permission.query - if it
works for gDM, it would work for gUM
Elad: I'll explore this, although I think the embargo issue
will be problem
harald: the user operates with the UA & OS separately, and the
UA and OS aren't friends - there needs to be a way to guide the
user toward the OS
… a query based system can only tell you about the situation as
it is now; the error feels like a superior situation
Elad: permission.query is async - so indeed the answer may no
longer the right one
[skipping issue 219]
[22]Solve user agent camera/microphone double-mute
(mediacapture-extensions)
[22] https://github.com/w3c/mediacapture-extensions/issues/39
[23][Slide 25]
[23]
https://lists.w3.org/Archives/Public/www-archive/2023Dec/att-0002/WEBRTCWG-2023-12-12__1_.pdf#page=25
[24][Slide 26]
[24]
https://lists.w3.org/Archives/Public/www-archive/2023Dec/att-0002/WEBRTCWG-2023-12-12__1_.pdf#page=26
[25][Slide 27]
[25]
https://lists.w3.org/Archives/Public/www-archive/2023Dec/att-0002/WEBRTCWG-2023-12-12__1_.pdf#page=27
[26][Slide 28]
[26]
https://lists.w3.org/Archives/Public/www-archive/2023Dec/att-0002/WEBRTCWG-2023-12-12__1_.pdf#page=28
[27][Slide 29]
[27]
https://lists.w3.org/Archives/Public/www-archive/2023Dec/att-0002/WEBRTCWG-2023-12-12__1_.pdf#page=29
[28][Slide 30]
[28]
https://lists.w3.org/Archives/Public/www-archive/2023Dec/att-0002/WEBRTCWG-2023-12-12__1_.pdf#page=30
[29][Slide 31]
[29]
https://lists.w3.org/Archives/Public/www-archive/2023Dec/att-0002/WEBRTCWG-2023-12-12__1_.pdf#page=31
[30][Slide 32]
[30]
https://lists.w3.org/Archives/Public/www-archive/2023Dec/att-0002/WEBRTCWG-2023-12-12__1_.pdf#page=32
[jan-ivar points to [31]w3c/mediacapture-main#982 ]
[31] https://github.com/w3c/mediacapture-main/issues/982
[32][Slide 33]
[32]
https://lists.w3.org/Archives/Public/www-archive/2023Dec/att-0002/WEBRTCWG-2023-12-12__1_.pdf#page=33
[33][Slide 34]
[33]
https://lists.w3.org/Archives/Public/www-archive/2023Dec/att-0002/WEBRTCWG-2023-12-12__1_.pdf#page=34
[jan-ivar points to [34]https://github.com/w3c/mediasession/
issues/279]
[34] https://github.com/w3c/mediasession/issues/279]
[hta, aboba +1 MuteReason on chat]
jan-ivar: the 1st thing we should is to make it clear where the
discussion is happening on github
… I didn't feel like the slides were representative of the
discussion on github
… I see the same issue with MuteReason as what I described
earlier
… I don't think we should expose an OS setting
… there are cases where the UA might be muting - which is why I
opened an issue to explore that question in [35]w3c/
mediacapture-main#982 which shows different interpretation by
UAs of muted state
[35] https://github.com/w3c/mediacapture-main/issues/982
Elad: I want to impress on everybody that this is an important
issue for users and developers, and brought critique on
alternative solutions that had been suggested
Youenn: there is an existing solution with the MediaSession API
that is shipping in Chrome
… we need to discuss with them whether this will solve this
issue or not, and if not, we need to understand the differences
… it may be that we could fix or extend the mediasession API
… we need to understand the relationship between the two no
matter what
… I also agree with Jan-Ivar we need to clarify the meaning of
mute vs end - this is hurting developers
… this feels as important as this issue
… there are interoperability issues in end vs mute - they're
all different across browsers
… I would like to start a discussion with the Media WG to
understand the interactions between track.muted and
MediaSession
Elad: we've looked into this and it didn't look like a
compelling solution
Bernard: when the app controls mute, it can inform the user
you're speaking while muted
… but it can't do that when the OS is in control; what would
you do with MuteReason?
Youenn: the track would be muted, but you would still be able
through a separate event to notify the app that the user is
speaking
Elad: but if the user is not trying to speak, the user would
still not know they're OS-muted in the app UI
Bernard: how would you surface this?
Elad: the app could reflect that via multiple UI states, or
reflect the latest detected change, or give a bit more context
in the UI
HTA: I tried in Chrome: setMicrophoneActive: true doesn't
unmute, setMicrophoneActive: false doesn't mute
… re end vs mute - if they're diverging behavior, we should fix
that - in implementations or specs, as needed
… but this is orthogonal
Jan-Ivar: the app receives enabled/muted already - MuteReason
doesn't address that
… the MediaSession API provides an API for applications that
have mute toggles to expand those to keyboard, hardware, lock
screen, etc
… there is no muting in MediaSession at the moment
… the UA is already allowed to mute at any point
Youenn: we do need to talk about the intents of the
MediaSession API
Elad: we've shown a PR of what MuteReason could do; we haven't
heard why we shouldn't expose an OS setting
… our own privacy review has qualified this as benign
… I suspect solving this with MediaSession will be complex, but
happy to be proved wrong
… Hope we can make progress on this before the next meeting
[36]Dynamic Switching in Captured Surfaces
[36] https://github.com/w3c/mediacapture-screen-share/issues/255
[37][Slide 37]
[37]
https://lists.w3.org/Archives/Public/www-archive/2023Dec/att-0002/WEBRTCWG-2023-12-12__1_.pdf#page=37
[38][Slide 38]
[38]
https://lists.w3.org/Archives/Public/www-archive/2023Dec/att-0002/WEBRTCWG-2023-12-12__1_.pdf#page=38
[39][Slide 39]
[39]
https://lists.w3.org/Archives/Public/www-archive/2023Dec/att-0002/WEBRTCWG-2023-12-12__1_.pdf#page=39
[40][Slide 40]
[40]
https://lists.w3.org/Archives/Public/www-archive/2023Dec/att-0002/WEBRTCWG-2023-12-12__1_.pdf#page=40
[41][Slide 41]
[41]
https://lists.w3.org/Archives/Public/www-archive/2023Dec/att-0002/WEBRTCWG-2023-12-12__1_.pdf#page=41
[42][Slide 42]
[42]
https://lists.w3.org/Archives/Public/www-archive/2023Dec/att-0002/WEBRTCWG-2023-12-12__1_.pdf#page=42
[43][Slide 43]
[43]
https://lists.w3.org/Archives/Public/www-archive/2023Dec/att-0002/WEBRTCWG-2023-12-12__1_.pdf#page=43
jan-ivar: good summary of the discussion, thanks! looking at
slide 40, would sourceswitch fire if there is no opt-in?
Trove: I think we could
jan-ivar: would the event come with the same stream in that
situation?
Trove: it will be a new stream
Jan-Ivar: I'm in favor of the late decision model: you get the
event regardless you opted in to assisted switching
… having the injection model as the default, and preventing it
with preventDefault() is aligned with well-known Web platform
patterns
Elad: looking at the code slide 42, I find it hard to
understand what preventDefault() does
… and if there is not preventDefault(), it's even less clear
that something default is happening
… this feels foot-gunny
… what purpose does this serve? What app needs to make a late
decision? it feels like app would always want one or the other
model
Youenn: I wasn't sure whether surface switching with injection
model was good or not
… I think it's good to support surface type change - we have
the configuration event for that
… allowing apps to optimize it is good; late-switching is a
nice-to-have, but not required if it's particularly difficult
to implement
… but having web developers flexibility is nice
Elad: you've often mentioned that some complexity is to the
detriment of the developer, which seems to be the case here
Jan-Ivar: the complexity comes from the fact that this is
changing an existing shipped behavior - we can't avoid it
… no matter what solution we choose, the default behavior won't
be obvious
… I think it's better to fall back on the injection model
Trove: it's hard to know *when* you need to switch tracks
… it affects capabilities, it may affect which methods can be
called
jan-ivar: the question is when the decision happens - the app
chooses
… a late decision can allow to minimize the glitch by detecting
what kind of replacement is happening
Elad: shouldn't the UA fix that?
jan-ivar: the UA can't fix this - e.g. with replaceTrack
because of downstream consequences e.g. in MediaRecorder
Elad: could we demonstrate a path forward for a later addition
of late decision?
Jan-Ivar: -1
[44]RtpSender Encoded Source
[44] https://github.com/w3c/webrtc-encoded-transform/issues/211
[45][Slide 46]
[45]
https://lists.w3.org/Archives/Public/www-archive/2023Dec/att-0002/WEBRTCWG-2023-12-12__1_.pdf#page=46
Slide [47]
[47]
https://lists.w3.org/Archives/Public/www-archive/2023Dec/att-0002/WEBRTCWG-2023-12-12__1_.pdf#page=54
Slide [48]
[48] https://github.com/w3c/webrtc-encoded-transform/issues/215
Slide [49]
[49]
https://lists.w3.org/Archives/Public/www-archive/2023Dec/att-0002/WEBRTCWG-2023-12-12__1_.pdf#page=55
Slide [50]
[50]
https://lists.w3.org/Archives/Public/www-archive/2023Dec/att-0002/WEBRTCWG-2023-12-12__1_.pdf#page=56
Slide [51]
[51]
https://lists.w3.org/Archives/Public/www-archive/2023Dec/att-0002/WEBRTCWG-2023-12-12__1_.pdf#page=57
Youenn: in general, we want to design the API to allow plugging
an encoder
… then we add specific constraints for cases where sources and
encoders are combined
… that would be my general advice
… I would be in favor to have immutable objects in general
Guido: so constructor over setMetadata?
Youenn: yes - that has been the approach with WebCodecs
Guido: I can agree with that
… For the forwarding use case, you already have frames that you
are forwarding
Harald: we have an agreed upon use case, the fan out, but we
don't have one for encoders
… I kind of agree treating frames as immutable is better if we
can solve the @@@ issue
… but for this use case, handling RTP-relevant data is what
matters, so I don't see the need for a new object
Jan-Ivar: we've heard many proposals for ways to expose the
media pipeline to JS
… via transforms
Guido: this is not for encoding, but forwarding already encoded
frames
Jan-Ivar: but does this not also allow JS to get frames from
anywhere?
Guido: from another PC?
Jan-Ivar: Youenn's proposal allowed to get frames from
WebCodecs
Guido: yes, but in that case they don't have the RTP metadata
that would allow forwarding
Jan-Ivar: but I'm not sure there is consensus on Youenn's
proposal
… re [slide 48], where would be RTCRtpSenderEncodedSource
exposed?
Guido: I think Youenn meant to expose on Worker only, with the
handle being transferable
Jan-Ivar: happy to hear that
[TimP, Harald gives +1 to the approach]
Bernard: the RTCRTpSenderEncodedSource - do we really need two
enqueue methods?
Guido: we only need one for the forwarding use case
Bernard: can we convert between audiochunk and and rtpchunk?
… this could simplify the API
Guido: there isn't such a way at the moment, but we could look
into one
Harald: a constructor with a chunk as input could do this, and
avoids touching rtpsenderencodedsource
Jan-Ivar: I think it may be a good direction, but I would love
to see JS code that shows where the encoded frames are coming
from
Guido: [shows slide 47]
Jan-Ivar: is there a receiver equivalent to this?
Guido: that would be the logical follow up to this
… e.g. to turn multiple input into a more reliable input for
playback
RESOLUTION: seems like a promising direction for which to see a
more complete proposal
[46]Keyframe API
[46] https://github.com/w3c/webrtc-encoded-transform/pull/215
[47][Slide 54]
[47]
https://lists.w3.org/Archives/Public/www-archive/2023Dec/att-0002/WEBRTCWG-2023-12-12__1_.pdf#page=54
Harald: [48]#215 is merged - we now have a spec for a keyframe
event
[48] https://github.com/w3c/webrtc-encoded-transform/issues/215
[49][Slide 55]
[49]
https://lists.w3.org/Archives/Public/www-archive/2023Dec/att-0002/WEBRTCWG-2023-12-12__1_.pdf#page=55
[50][Slide 56]
[50]
https://lists.w3.org/Archives/Public/www-archive/2023Dec/att-0002/WEBRTCWG-2023-12-12__1_.pdf#page=56
[51][Slide 57]
[51]
https://lists.w3.org/Archives/Public/www-archive/2023Dec/att-0002/WEBRTCWG-2023-12-12__1_.pdf#page=57
Jan-Ivar: +1 - hoping we can present API proposals next time
around
Summary of resolutions
1. [52]Consensus to specify an interoperable behavior and
iterate initially on a pull request to be proposed by Elad
2. [53]seems like a promising direction for which to see a
more complete proposal
Minutes manually created (not a transcript), formatted by
[54]scribe.perl version 221 (Fri Jul 21 14:01:30 2023 UTC).
Received on Wednesday, 13 December 2023 07:50:09 UTC