Re: [minutes] June 18 meeting

Video recording:

https://youtu.be/2dWJN73CwkY


On Wed, Jun 19, 2024 at 8:38 AM Dominique Hazael-Massieux <dom@w3.org>
wrote:

>
> Hi,
>
> The minutes of our meeting held yesterday (June 18) are available at:
>    https://www.w3.org/2024/06/18-webrtc-minutes.html
>
> and copied as text below.
>
> Dom
>
>                        WebRTC June 18 2024 meeting
>
> 18 June 2024
>
>     [2]Agenda. [3]IRC log.
>
>        [2] https://www.w3.org/2011/04/webrtc/wiki/June_18_2024
>        [3] https://www.w3.org/2024/06/18-webrtc-irc
>
> Attendees
>
>     Present
>            Bernard, Dom, FlorentCastelli, Harald, Jan-Ivar,
>            JianjunZhu, PatrickRockhill, PeterThatcher, Sameer,
>            SunShin, TimP, TonyHerre, Youenn
>
>     Regrets
>            -
>
>     Chair
>            Bernard, HTA, Jan-Ivar
>
>     Scribe
>            dom
>
> Contents
>
>      1. [4]Future meetings
>      2. [5]Proposal: merging more extensions in WebRTC Rec
>      3. [6]WebRTC Charter Renewal
>      4. [7]Media Capture Extensions
>           1. [8]Issue #145 Consider adding onVoiceActivity event on
>              MediaStreamTrack for audio
>           2. [9]Issue #149 How to select camera presets that have
>              better power efficiency at the expense of quality?
>      5. [10]ICE improvements: send and prevent ICE check over a
>         candidate pair
>      6. [11]WebRTC: PC.local_description and friends - snapshot
>         views or dynamic views?
>      7. [12]Should devicechange fire when the device info changes?
>      8. [13]Summary of resolutions
>
> Meeting minutes
>
>     Slideset: [14]https://lists.w3.org/Archives/Public/www-archive/
>     2024Jun/att-0002/WEBRTCWG-2024-06-18.pdf
>
>       [14]
>
> https://lists.w3.org/Archives/Public/www-archive/2024Jun/att-0002/WEBRTCWG-2024-06-18.pdf
>
>    Future meetings
>
>     Bernard: there has been discussion of canceling our scheduled
>     meeting on July 16 and instead meet on August 27
>     … any objection to that plan?
>
>     [none expressed]
>
>     RESOLUTION: cancel July 16 meeting and schedule one on Aug 27
>
>    [15]Proposal: merging more extensions in WebRTC Rec
>
>       [15] https://github.com/w3c/webrtc-charter/pull/83
>
>     [16][Slide 10]
>
>       [16]
>
> https://lists.w3.org/Archives/Public/www-archive/2024Jun/att-0002/WEBRTCWG-2024-06-18.pdf#page=10
>
>     Dom: any comment or objection to the proposal?
>
>     Jan-Ivar: agree on the extension spec being confusing
>     … this isn't a charter change, right?
>
>     Dom: correct
>
>     Jan-Ivar: no objection, wonder if this could be used to
>     mediacapture-main pre-Rec
>
>     Dom: yes, the process would allow for that
>
>     Bernard: what does reasonable test coverage encompass?
>
>     Dom: Basically enough to have confidence the implementation
>     reasonably matches what we've specified
>
>     TimP: should we consider non-browser implementations in this
>     context?
>
>     Dom: That may be something to consider independently of this
>     policy; but my personal sense is that this WG should focus on
>     browser interop, even though other implementation contexts are
>     a great secondary impact of this work
>
>     Jan-Ivar: +1
>
>     Youenn: what constitutes a commitment? in some cases, it may be
>     we're willing to implement but with unclear or a moving
>     implementation roadmap
>
>     Dom: commitment could be a bug tracking link, a standards
>     position from an implementor
>
>     RESOLUTION: proceed with adopting new merge-guide guidance
>
>    [17]WebRTC Charter Renewal
>
>       [17]
> https://lists.w3.org/Archives/Public/public-webrtc/2024May/0031.html
>
>     [18][Slide 11]
>
>       [18]
>
> https://lists.w3.org/Archives/Public/www-archive/2024Jun/att-0002/WEBRTCWG-2024-06-18.pdf#page=11
>
>     Dom: any objection to proceeding with that plan?
>
>     RESOLUTION: Proceed with editorial renewal of the WebRTC WG
>     charter
>
>    [19]Media Capture Extensions
>
>       [19] https://github.com/w3c/mediacapture-extensions/
>
>      Issue [20]#145 Consider adding onVoiceActivity event on
>      MediaStreamTrack for audio
>
>       [20] https://github.com/w3c/mediacapture-extensions/issues/145
>
>     [21][Slide 15]
>
>       [21]
>
> https://lists.w3.org/Archives/Public/www-archive/2024Jun/att-0002/WEBRTCWG-2024-06-18.pdf#page=15
>
>     Jianjun: is there support for adding such an event?
>
>     Youenn: the "unmute microphone" hint is a great use case; the
>     media session API will help to fully mute the microphone, so
>     helping to expose that state to the end user is a great idea
>     … there is OS support for it already; so this is a valid use
>     case
>     … I'm not sure we can limit processing to only when the user is
>     actively speaking since it may be too late to do the processing
>     once the event has been received
>
>     Jan-Ivar: the two use cases may lead to different solutions; I
>     very strongly support the 1st use case, the 2nd may have
>     privacy implications
>     … I'm not even sure you need a constraint for that - maybe if
>     it's onerous for implementations to do that
>
>     Youenn: there might be use cases via audio stats, e.g. there is
>     an rtp header extension to expose voice activity as a boolean,
>     typically set by the encoder
>     … providing that in real-time may make sense, but not sure if
>     the event is the right approach, this will need more
>     experimenting
>
>     Bernard: in terms of privacy implications, determining
>     start/end of speech occurrences may reveal more information to
>     the app than the user would expect
>     … we could fire only "start" and limit it to a given interval
>     … for the "unmute hint" use case, you don't need to know how
>     long the person is speaking, and it doesn't need to be super
>     frequent
>
>     Jianjun: this works for the 1st use case, but wouldn't help for
>     the 2nd use case where you need to know when to stop and start
>     processing
>
>     Tony: would a simple boolean suffice to give confidence on when
>     to show the hint? e.g. clearly speaking vs maybe speech in the
>     background
>
>     Jianjun: not sure - today looking for support for having such
>     an API or not
>
>     TimP: we need to be clearer on whether this is voice detection
>     or audio levels - they come with different requirements
>     … the 1st one is specifically about voice, and the more
>     specific the better
>     … the 2nd one would want to work with singing, groups of
>     people, etc
>     … this feels like 2 APIs
>
>     Jianjun: I'm hearing the 2nd use case needs more thinking
>
>     Jan-Ivar: +1 on these being separate use cases that require
>     separate situations
>     … The 1st one is a single event that fires when the microphone
>     is muted and there is voice activity
>     … The 2nd one might be too noisy an an event on the main
>     thread; it feels like metadata to attach to the audio, outside
>     of the main thread
>
>     Jianjun: I'm happy to focus on the 1st use case atm
>
>     Youenn: +1 to Jan-Ivar; 1st use cases focused on muted track,
>     2nd on unmuted; the 2nd could be in audio stats, although it
>     may be more usefully exposed in the audio worklet
>     … in any case, separating them, doing the 1st one quickly and
>     taking more time on the audio processing optimization
>
>     Harald: you could already skip processing audio frames with
>     only zeros, although that itself introduces some overhead
>
>     Jianjun: I'm hearing we should focus on the 1st use case, and
>     look into how to solve the 2nd use case separately
>
>     RESOLUTION: proceed with a pull request for the 1st use case,
>     and open a separate issue for optimizing audio processing
>
>      Issue [22]#149 How to select camera presets that have better power
>      efficiency at the expense of quality?
>
>       [22] https://github.com/w3c/mediacapture-extensions/issues/149
>
>     [23][Slide 16]
>
>       [23]
>
> https://lists.w3.org/Archives/Public/www-archive/2024Jun/att-0002/WEBRTCWG-2024-06-18.pdf#page=16
>
>     [24][Slide 17]
>
>       [24]
>
> https://lists.w3.org/Archives/Public/www-archive/2024Jun/att-0002/WEBRTCWG-2024-06-18.pdf#page=17
>
>     Dom: note that afaict there are no implementations of
>     powerEfficientPixelFormat, so we could replace it altogether at
>     this stage without backwards compat
>
>     Bernard: would powerEfficient always include the power
>     efficient pixel format? if not, it may be useful to retain the
>     granularity
>
>     Harald: this reminds me of content-hint - you don't want to
>     specify exactly what to do, but pushing the UA in a certain
>     direction
>     … pixelFormat feel too constrainted, a more general one to
>     replace it sounds like a good idea
>     … e.g. a powerQualityTradeoff constraint
>
>     Bernard: the pixelFormat feels different than the general power
>     efficiency
>
>     TimP: any reason the default should not be "powerEfficient"?
>
>     Youenn: this may break existing expectations e.g. for bar code
>     scanners, high quality podcast recordings
>     … it may be ideal to have that constraint default to true
>     eventually, but that may be hard to do in a backward compatible
>     manner, at least initially
>     … I'm hearing support for removing powerEfficientPixelFormat
>     and move toward a more general powerEfficient constraint
>
>     Jianlin: if two pages request a camera, one with powerEfficient
>     and the other not, what would happen?
>
>     Youenn: e.g. the UA could re-size the video, with a bit of a
>     quality loss
>     … My thinking is "short usage, quality; longer, power
>     efficiency"
>
>     Jan-Ivar: impact of power efficiency on barcode scanning?
>
>     Youenn: e.g. the autofocus might be more finicky in power
>     efficient setting
>
>     Guido: we could start by adding a new constraint and reconsider
>     removal of pixelFormat later - we don't have to do that at the
>     same time
>
>     RESOLUTION: Proceed with a pull request to add a powerEfficient
>     constraint
>
>    [25]ICE improvements: send and prevent ICE check over a candidate pair
>
>       [25] https://github.com/w3c/webrtc-extensions/issues/209
>
>     [26][Slide 20]
>
>       [26]
>
> https://lists.w3.org/Archives/Public/www-archive/2024Jun/att-0002/WEBRTCWG-2024-06-18.pdf#page=20
>
>     [27][Slide 21]
>
>       [27]
>
> https://lists.w3.org/Archives/Public/www-archive/2024Jun/att-0002/WEBRTCWG-2024-06-18.pdf#page=21
>
>     Bernard: is consent freshness included in your concept of ICE
>     check here?
>     … if so, it may raise security issues
>
>     Sameer: I'll try to address that in the discussion of the API
>     … the API I want to propose addresses connectivity checks,
>     keepalive, and RTT determination, while conserving bandwidth &
>     power
>
>     [28][Slide 22]
>
>       [28]
>
> https://lists.w3.org/Archives/Public/www-archive/2024Jun/att-0002/WEBRTCWG-2024-06-18.pdf#page=22
>
>     Bernard: re consent freshness: is it considered a triggered
>     check? it's good that responses can't be prevent, but can the
>     app prevent consent freshness to go out?
>
>     Sameer: don't know off the top of my head, but +1 on making
>     sure the app can't prevent it
>
>     Jan-Ivar: can you describe the use cases to expose the API to
>     JS?
>
>     Sameer: this is more detailed in the issue description on
>     github
>     … STUN checks are sent for keepalive on the selected pair every
>     2s or so, but only every 15s or so on non-selected pairs
>     … this API allows to monitor the quality of different candidate
>     pairs and possibly switch e.g. if the RTT is lower
>     … we've tried this out on mobile and seen very different RTT
>     over the life of a session, across different network interfaces
>     for instance
>
>     Jan-Ivar: there is no way for the UA to be doing this on its
>     own?
>
>     Sameer: there is no obligation to the UA to be monitoring
>     quality over ICE candidates
>
>     Jan-Ivar: could this be a IceTransport option instead though?
>     instead of lots of new API surface
>
>     Sameer: picking an alternative may be hard to specify as a
>     configuration (e.g. if you want to avoid relays due to latency)
>
>     Harald: the UA cannot know what options the application might
>     have available that are not immediately visible
>     … the purpose is the to allow the app to control the use of
>     connections and figure the information it can get
>     … the UA algorithms have to evolve slowly and work well in the
>     general case
>     … this API will allow experiemntation and considerations of
>     things the UA cannot know
>     … it's a movement towards ensuring the app can do what it wants
>     to do while making the default behavior work well
>
>     Peter: re consent checks - I think it would be fine to disable
>     consent checks, this would only mean that it would stop the ICE
>     transport
>     … (i.e. it wouldn't be unsafe, but a bit of a footgun)
>     … in terms of the why, it would be very difficult for the UA to
>     know what to do or to express it as a configuration
>
>     [29][Slide 23]
>
>       [29]
>
> https://lists.w3.org/Archives/Public/www-archive/2024Jun/att-0002/WEBRTCWG-2024-06-18.pdf#page=23
>
>     [30][Slide 24]
>
>       [30]
>
> https://lists.w3.org/Archives/Public/www-archive/2024Jun/att-0002/WEBRTCWG-2024-06-18.pdf#page=24
>
>     Sameer: the event-based API doesn't tell you when a check was
>     sent if initiated by the ICE agent - the app may thus get a
>     failure it tries to send a check too soon after the ICE agent
>
>     [31][Slide 25]
>
>       [31]
>
> https://lists.w3.org/Archives/Public/www-archive/2024Jun/att-0002/WEBRTCWG-2024-06-18.pdf#page=25
>
>     TimP: there seems to be a big difference in scope, in terms of
>     the context of what the app will see
>     … the promises carries a lot of the context the event doesn't
>     … e.g. whether the response comes from its own request or not -
>     that feels like useful information to have - maybe this could
>     be added to the event as well
>
>     Sameer: the app could determine whether they sent it - if they
>     got a failure trying to request a check, they can know the one
>     in flight comes from the agent
>     … since there can only be one check in flight for a given pair
>     … we could also expose the transaction id in the return value
>     of the request (since it is exposed in the event)
>
>     Jan-Ivar: RTCIceCandidatePair is now an interface - the check
>     method could be moved there
>     … what is the transactionId array buffer
>
>     Sameer: it's the transaction id of the ICE check itself
>
>     Peter: it's 20 bytes in the STUN header that is useful for
>     debugging, it's unique to each check
>
>     Jan-Ivar: so the proposal is a check method that the UA has
>     done an ICE check on behalf of the app; could you cause network
>     spam by running this in a loop?
>
>     Sameer: rate limiting is called out; since the main use case is
>     to determine RTT / inactive pairs, 1s interval should be
>     sufficient
>
>     Jan-Ivar: I would go with the promise approach - the event
>     approach seems to be carrying a lot of state, which goes
>     against general API advice
>
>     Peter: what happens if an ICE check goes lost? is there a
>     timeout to determine it is no longer in flight?
>
>     Sameer: when a check is sent out, it's marked as in progress;
>     if it timeouts, it's marked as failed
>
>     Peter: it doesn't allow the app to decide what the timeout
>     should be; and we would want more than one check in flight in
>     general as during establishments
>     … as long as the browser is exposing rate limits, this should
>     be OK
>
>     Jan-Ivar: re event API shape, maybe the values could be exposed
>     on the icecandidatepair instead of the event
>
>     Sameer: this would be similar to what we can get with getStats
>     - I think it's more useful to expose immediate feedback with
>     the response
>
>     Sameer: overall, I'm hearing support for the promises approach,
>     and investigating multiple in-flight checks at the same time
>
>    [32]WebRTC: PC.local_description and friends - snapshot views or
>    dynamic views?
>
>       [32] https://github.com/w3c/webrtc-pc/issues/2977
>
>     [33][Slide 29]
>
>       [33]
>
> https://lists.w3.org/Archives/Public/www-archive/2024Jun/att-0002/WEBRTCWG-2024-06-18.pdf#page=29
>
>     RESOLUTION: no objection to proposal to align spec with
>     implementations
>
>    [34]Should devicechange fire when the device info changes?
>
>       [34] https://github.com/w3c/mediacapture-main/issues/966
>
>     [35][Slide 30]
>
>       [35]
>
> https://lists.w3.org/Archives/Public/www-archive/2024Jun/att-0002/WEBRTCWG-2024-06-18.pdf#page=30
>
>     [36][Slide 31]
>
>       [36]
>
> https://lists.w3.org/Archives/Public/www-archive/2024Jun/att-0002/WEBRTCWG-2024-06-18.pdf#page=31
>
>     TimP: how about most recently inserted device?
>
>     Jan-Ivar: for it to be a strong signal, it has to be recent
>     … also there can be several devices (e.g. a camera that sports
>     a mic)
>     … but we can bikeshed the name on the issue
>     … or are you thinking of a different logic?
>
>     TimP: mostly suggesting a way to simplify the explanation of
>     the behavior
>
>     Jan-Ivar: inserted may have connotations that are no longer
>     accurate for all devices, but has a nice intent aspect to it
>
>     harald: it's a new feature - how can you detect it?
>
>     Jan-Ivar: you get an empty array if no devices were inserted
>     (vs no property if not implemented)
>
>     RESOLUTION: Continue discussion on PR towards merging
>
> Summary of resolutions
>
>      1. [37]cancel July 16 meeting and schedule one on Aug 27
>      2. [38]proceed with adopting new merge-guide guidance
>      3. [39]Proceed with editorial renewal of the WebRTC WG charter
>      4. [40]proceed with a pull request for the 1st use case, and
>         open a separate issue for optimizing audio processing
>      5. [41]Proceed with a pull request to add a powerEfficient
>         constraint
>      6. [42]no objection to proposal to align spec with
>         implementations
>      7. [43]Continue discussion on PR towards merging
>
>
>      Minutes manually created (not a transcript), formatted by
>      [44]scribe.perl version 221 (Fri Jul 21 14:01:30 2023 UTC).
>
>       [44] https://w3c.github.io/scribe2/scribedoc.html
>
>
>