[minutes] WebRTC February 2024 meeting

Hi,

The minutes of our Feb 2024 meeting held yesterday are available at:
   https://www.w3.org/2024/02/20-webrtc-minutes.html

and copied as text below.

Dom

                       WebRTC February 2024 meeting

20 February 2024

    [2]Agenda. [3]IRC log.

       [2] https://www.w3.org/2011/04/webrtc/wiki/February_20_2024
       [3] https://www.w3.org/2024/02/20-webrtc-irc

Attendees

    Present
           Bernard, Carine, Dom, Elad, Fippo, Florent, Guido,
           Harald, Jan-Ivar, PatrickRockhill, Peter, SunShin, TimP,
           Youenn

    Regrets
           -

    Chair
           Bernard, HTA, Jan-Ivar

    Scribe
           dom

Contents

     1. [4]WebRTC API
          1. [5]Issues with receive-only codecs aboba/
             hevc-webrtc#22
          2. [6]Modify the codec description model to ease
             describing changes #2925 #2935
          3. [7]Should media capabilities influence what is exposed
             in what is exposed in WebRTC offers and answers #2929
          4. [8]Existing setCodecPreferences NOTE is wrong and
             should be deleted #2933
          5. [9]setCodecPreferences to deal with both send and recv
             codecs #2939
     2. [10]Screen Capture
          1. [11]Issue triage and milestones
          2. [12]Should we have a screenshare extension spec? #297
          3. [13]Distinguish cancellations from absent OS
             permissions #281
     3. [14]MediaStream Recording
          1. [15]Issue #202 Deprecate isTypeSupported
     4. [16]Media Capture specs
          1. [17]Issue triage and milestones of Media Capture and
             Streams
          2. [18]captureStream on OffscreenCanvas on
          3. [19]Expose MediaStream in Workers
          4. [20]How should MediaStreamTrack interact with BFCache?
          5. [21]Add guidance for defining a new source of
             MediaStreamTrack
     5. [22]Summary of resolutions

Meeting minutes

    Slideset: [23]https://lists.w3.org/Archives/Public/www-archive/
    2024Feb/att-0008/WEBRTCWG-2024-02-20.pdf

      [23] 
https://lists.w3.org/Archives/Public/www-archive/2024Feb/att-0008/WEBRTCWG-2024-02-20.pdf

   [24]WebRTC API

      [24] https://github.com/w3c/webrtc-pc/

     Issues with receive-only codecs [25]aboba/hevc-webrtc#22

      [25] https://github.com/aboba/hevc-webrtc/issues/22

    [26][Slide 11]

      [26] 
https://lists.w3.org/Archives/Public/www-archive/2024Feb/att-0008/WEBRTCWG-2024-02-20.pdf#page=11

    Bernard: not a WebRTC issue - it came up in IETF AVTCORE wrt
    receive-only codecs

    Repository: [27]w3c/webrtc-pc

      [27] https://github.com/w3c/webrtc-pc/

     Modify the codec description model to ease describing changes
     [28]#2925 [29]#2935

      [28] https://github.com/w3c/webrtc-pc/issues/2925
      [29] https://github.com/w3c/webrtc-pc/issues/2935

    [30][Slide 12]

      [30] 
https://lists.w3.org/Archives/Public/www-archive/2024Feb/att-0008/WEBRTCWG-2024-02-20.pdf#page=12

    [31][Slide 13]

      [31] 
https://lists.w3.org/Archives/Public/www-archive/2024Feb/att-0008/WEBRTCWG-2024-02-20.pdf#page=13

    [32][Slide 14]

      [32] 
https://lists.w3.org/Archives/Public/www-archive/2024Feb/att-0008/WEBRTCWG-2024-02-20.pdf#page=14

    Harald: is this a reasonable direction to go in?

    Bernard: it is; the more I read, the more current situation
    doesn't make sense, with too much unspecified, so this is a
    good step forward

    Jan-Ivar: overall it makes sense to add more details to it and
    it will help having a more neutral baseline
    … we need to keep track of fingerprinting concerns, but we
    should be aligning with Media Capabilities in any case
    … this looks like an improvement to me

    Bernard: Media Capabilities doesn't have any way to look at
    directionality except encoder/decoder - not sure if it's a good
    match

    Harald: you have to make two calls to media capabilities to
    figure what you can send/received

    bernard: so you're confident that encoder/decoder matches
    send/receive? we should test it too

    Harald: hearing no push back, we'll bring this as candidate to
    merge at the editors meeting on Thursday

    RESOLUTION: Bring [33]#2935 for merging to editors meeting

      [33] https://github.com/w3c/webrtc-pc/issues/2935

     Should media capabilities influence what is exposed in what is
     exposed in WebRTC offers and answers [34]#2929

      [34] https://github.com/w3c/webrtc-pc/issues/2929

    [35][Slide 15]

      [35] 
https://lists.w3.org/Archives/Public/www-archive/2024Feb/att-0008/WEBRTCWG-2024-02-20.pdf#page=15

    Youenn: more and more codecs get exposed on the Web and in
    WebRTC
    … for playback, media capabilities allow to control how much
    information gets exposed to the web site
    … for WebRTC, everything gets exposed via getCapabilities or
    via SDP
    … this is a fingerprinting issue, but also has an impact on the
    number of allocated payload types

    Harald: not all codecs get exposed, but they can be discovered
    through setLocalDescription

    Youenn: getCapabilities was initially designed to expose
    everything supported, but it would be best to deprecate it
    … can we move to use media capabilities as a replacement?

    [36][Slide 16]

      [36] 
https://lists.w3.org/Archives/Public/www-archive/2024Feb/att-0008/WEBRTCWG-2024-02-20.pdf#page=16

    Youenn: please chime in in [37]#2929 too

      [37] https://github.com/w3c/webrtc-pc/issues/2929

    Bernard: this makes sense for the simpler codecs à la VP8, VP9,
    AV1, but e.g. HEVC raises weird situations
    … say I query a common level that can be used for
    encoding/decoding, how would this impact the whole offer/answer
    negotiation?

    Youenn: the idea would be that media capabilities would give
    you WebRTC-specific data that could be plugged into the WebRTC
    API

    Bernard: but for these send-only or receive-only situations
    across levels...

    Youenn: SDP can't express all these situations
    … this wouldn't be an improvement, but it doesn't make things
    worse
    … and at least, the web app would know what's not available
    e.g. on the decoding side

    TimP: I like this - it feels overdue
    … the UA-to-UA disadvantage doesn't feel too serious
    … the only issue I have is how precise the query would have to
    be e.g. on which submode of which profile of a codec
    … esp since UA are known to lie on some of these questions

    Bernard: indeed, e.g. with H264 and 265

    Youenn: this profile complexity would be in scope indeed

    Harald: it's probably good to link webrtc and media
    capabilities codec more closely
    … Youenn has a PR to have media capabilities return a WebRTC
    shape for doecs when queried
    … to ensure it is inspectable and usable in WebRTC land
    … it's good to have the default set of codecs be implementation
    defined
    … if we have a codec that is universally supported by a given
    platform, it doesn't expose much fingerprinting surface to
    expose it

    Youenn: I'll revive the Media Capabilities proposal

    Jan-Ivar: I'm also supportive; reducing fingerprinting surface
    feels good, or at least reducing the list to a default list

    Youenn: I'll work on an API proposal to tie Media Capabilities
    and WebRTC - not sure yet if it's at the transceiver level,
    please bring input on the issue

    Harald: the codec model description will help as well for this

    RESOLUTION: Proceed with proposal based on data-minimization
    appraoch

    RESOLUTION: Youenn to revive to Media Capabilities proposal
    [38]media-capabilities#186

      [38] https://github.com/w3c/media-capabilities/issues/186

     Existing setCodecPreferences NOTE is wrong and should be deleted
     [39]#2933

      [39] https://github.com/w3c/webrtc-pc/issues/2933

    [40][Slide 17]

      [40] 
https://lists.w3.org/Archives/Public/www-archive/2024Feb/att-0008/WEBRTCWG-2024-02-20.pdf#page=17

    Fippo: if no objection, I'll bring a PR and a WPT test

    Bernard: I agree it's extraneous

    RESOLUTION: Fippo to submit a PR to remove note and a matching
    WPT test

     setCodecPreferences to deal with both send and recv codecs [41]#2939

      [41] https://github.com/w3c/webrtc-pc/issues/2939

    [42][Slide 18]

      [42] 
https://lists.w3.org/Archives/Public/www-archive/2024Feb/att-0008/WEBRTCWG-2024-02-20.pdf#page=18

    [43][Slide 19]

      [43] 
https://lists.w3.org/Archives/Public/www-archive/2024Feb/att-0008/WEBRTCWG-2024-02-20.pdf#page=19

    Bernard: I'm not sure this is entirely right given that it's
    based on a JSEP paragraph that looks like it may be wrong
    … sendrecv, sendonly, recvonly are 3 separate sets that
    setCodecPreferences gives order to
    … when changing direction, you're shifting to a different set
    … it's not clear that the preferences can survive such a shift

    Fippo: JSEP is confusing in that area
    … it only talks about sendrecv, not the two other sets you're
    alluding to
    … it may be challenging to change this in terms of web
    compatibility though

    Jan-Ivar: regardless of what JSEP says, it would be desirable
    if we kept sCP and direction change orthogonal
    … this could be addressed by having a super list that is
    filtered; if the filter ends up with an empty list, I'm a bit
    wary about throwing, although this does feel like a mistake

    Harald: changing direction after you have configured the codec
    influences the set of codecs available
    … if we require at least one sendrecv codec, we are safe, and
    adding recvonly codecs is also safe
    … this is a painful aspect of SDP - we should just admit
    failure on the sendonly case and figure out the least painful
    approach to deal with - maybe the last option

    Fippo: if we have nothing in common, the regular approach would
    be to reject the m-line, hence my preference for the 3rd option
    … should we wait for a JSEP change in this space?

    Bernard: I think we should come up with a proposal for JSEP

    Fippo: OK, will bring this back at the next meeting after more
    off-line discussions

   [44]Screen Capture

      [44] https://github.com/w3c/mediacapture-screen-share/

    [45][Slide 22]

      [45] 
https://lists.w3.org/Archives/Public/www-archive/2024Feb/att-0008/WEBRTCWG-2024-02-20.pdf#page=22

    Bernard: the WPT results for Screen Capture shows little
    adoption for capture controller

    Jan-Ivar: lots of green (getDisplayMedia works in all
    browsers), the red parts are for the new capture controller API
    … some red on iframe delegation, but the big difference is
    capture controller
    … it's not controversial, but hasn't been implemented outside
    of Chrome yet
    … it's not on FF's short term roadmap

    Youenn: likewise for Safari
    … we discussed a similar issue in Media WG - moving to CR would
    still be beneficial

    Jan-Ivar: +1

     Issue triage and milestones

    [46][Slide 24]

      [46] 
https://lists.w3.org/Archives/Public/www-archive/2024Feb/att-0008/WEBRTCWG-2024-02-20.pdf#page=24

    Jan-Ivar: please chime in if you feel the assignment of issues
    to milestones need adjustment

     Should we have a screenshare extension spec? [47]#297

      [47] https://github.com/w3c/mediacapture-screen-share/issues/297

    [48][Slide 25]

      [48] 
https://lists.w3.org/Archives/Public/www-archive/2024Feb/att-0008/WEBRTCWG-2024-02-20.pdf#page=25

    Youenn: I think it makes sense; in the Media WG, it was advised
    that an extension spec is more complex and needs more work for
    editors, but from the point of view for web developers, it
    clarifies what is mature and what isn't
    … if it's not useful for developers, then maybe a forever CR
    would be a better model

    TimP: do we think realistically it will help? are the 12
    enhancements really blocking? are the 19 issues solvable in a
    reasonable timeframe?

    Jan-Ivar: I think it will help because the enhancement are real
    additional features

    Elad: +1 if it helps with making progress on issues

    TimP: but do we thinkg the 19 CR issues are solvable in a
    sensible timeframe?

    Jan-Ivar: yes

    Elad: there may also be issues that shouldn't be considered as
    CR blocking

    Jan-Ivar: getDisplayMedia mostly works, and separating what's
    additional value would help

    Bernard: +1 that they're addressable, and they would allow to
    get e.g. more focus on privacy issues

    Jan-Ivar: hearing mostly agreement this would be the right path
    forward

    Elad: when do we want to lock the list of issues as
    CR-blocking?

    Jan-Ivar: I think we can start moving issues over to the new
    repo right away, and move them back if needed
    … and then we have to do the work to close the 19 issues

    RESOLUTION: Create extensions repo for screen capture and move
    issues related to new features there

     Distinguish cancellations from absent OS permissions [49]#281

      [49] https://github.com/w3c/mediacapture-screen-share/issues/281

    [50][Slide 26]

      [50] 
https://lists.w3.org/Archives/Public/www-archive/2024Feb/att-0008/WEBRTCWG-2024-02-20.pdf#page=26

    Elad: this is a problem worth solving; this sounds like a
    possible solution, wonder if we could improve it
    … i.e. the spec isn't explicit on this, would be useful to make
    it so
    … also not sure NotFoundError isn't the most explicit
    expression
    … e.g. we could define a new error with a more specific type

    Jan-Ivar: in general, there is generally pushback against
    defining new custom errors
    … and prefer re-using existing DOM errors even if they're not a
    perfect fit
    … I'm not too concerned about this imperfect fit

    Youenn: NotFoundError feels like a reasonable minimal API
    already in use; in terms of ergonomics, I think it's ok as long
    as the spec is very explicit about this interpretation of
    NotFoundError

    Jan-Ivar: +1 to making this explicit in the spec

    Elad: can we make it that NotFoundError be restricted to this?
    e.g. in a situation where constraints would limit shareable
    surfaces?

    Jan-Ivar: it sounds like NotFoundError would still be a
    reasonable fit for that situation, but that feels like an edge
    case

    Elad: fair, we can leave that question aside
    … could we still allow UA-dependent subclassing?

    Dom: that would lead to non-interoperable behavior

    Youenn: re OS permissions, I'm not sure that's a well-defined
    concept for the platform
    … we should check what's being used e.g. in HTML or Permissions

    Jan-Ivar: maybe this is a clarification to bring to
    mediacapture-screenshare on "no sources of type T are
    available" with a parenthesis that describes these examples

    Jan-Ivar: would we want to apply to this mic/cameras? knowing
    that they could be distinguished from no hardware with
    enumerateDevices()

    Guido: for OS permissions in camera/mic, we use notallowederror
    with a different message

    Elad: enumerateDevices() isn't 100% robust to make the
    distinction given that users can plug/unplug devices

    RESOLUTION: Move forward with a PR to clarify that
    NotFoundError would apply for OS permissions in screen-capture

    Jan-Ivar: I'll file an issue for a follow up discussion on
    mediacapture-main

    Elad: aligning the two would be best (although not fully
    required)

   [51]MediaStream Recording

      [51] https://github.com/w3c/mediacapture-record/

    [52][Slide 29]

      [52] 
https://lists.w3.org/Archives/Public/www-archive/2024Feb/att-0008/WEBRTCWG-2024-02-20.pdf#page=29

    Bernard: WPT shows tests with limited support

    Youenn: Safari doesn't support webm recording; not sure about
    mp4

    dom: I don't think there are codecs requirements in recording -
    if so, codec specific tests should be marked as optional

    [53][Slide 30]

      [53] 
https://lists.w3.org/Archives/Public/www-archive/2024Feb/att-0008/WEBRTCWG-2024-02-20.pdf#page=30

    Jan-Ivar: overall numbers are looking pretty good

    Youenn: not sure why there are codec/format-specific tests e.g.
    for stop()

    [54][Slide 31]

      [54] 
https://lists.w3.org/Archives/Public/www-archive/2024Feb/att-0008/WEBRTCWG-2024-02-20.pdf#page=31

    Bernard: I support this; WebCodecs is indeed the way forward
    for many of the requested enhancements for recorder

    [no objection raised to proceeding with that plan]

     Issue [55]#202 Deprecate isTypeSupported

      [55] https://github.com/w3c/mediacapture-record/issues/202

    [56][Slide 32]

      [56] 
https://lists.w3.org/Archives/Public/www-archive/2024Feb/att-0008/WEBRTCWG-2024-02-20.pdf#page=32

    Bernard: this could simply quote media capabilities spec

    Jan-Ivar: we could return a fixed list as Youenn suggested
    earlier in the webrtc case
    … this would solve the privacy issue neatly here

    Bernard: wfm

    Youenn: +1

    Dom: it needs to stay in the spec for web compat, but should be
    described as returning fixed answers and its usage discouraged

    RESOLUTION: Make isTypeSupported return a fixed list of answers
    and mark it as deprecated

   Media Capture specs

     Issue triage and milestones of Media Capture and Streams

    [57][Slide 35]

      [57] 
https://lists.w3.org/Archives/Public/www-archive/2024Feb/att-0008/WEBRTCWG-2024-02-20.pdf#page=35

    Jan-Ivar: we would like to see more activity in this spec to
    accelerate progress towards Rec

     [58]captureStream on OffscreenCanvas on

      [58] https://github.com/w3c/mediacapture-fromelement/issues/65

    [59][Slide 37]

      [59] 
https://lists.w3.org/Archives/Public/www-archive/2024Feb/att-0008/WEBRTCWG-2024-02-20.pdf#page=37

    Youenn: no objection - but feels like low priority, not sure
    there is much web developer demand for this

    Harald: offscreencavas has usages, not all of them in workers
    … this isn't related to mediacapture-main though?
    … the only relation is the availability on MediaStream on
    workers

    Jan-Ivar: yes, that's the next issue I wanted to discuss, since
    answering yes to this would give an answer to this

    Jan-Ivar: not hearing objection, nor much interest either

    RESOLUTION: Supporting captureStream on OffscreenCanvas is
    reasonable but low priority

     [60]Expose MediaStream in Workers

      [60] https://github.com/w3c/mediacapture-extensions/pull/26

    [61][Slide 38]

      [61] 
https://lists.w3.org/Archives/Public/www-archive/2024Feb/att-0008/WEBRTCWG-2024-02-20.pdf#page=38

    Jan-Ivar: not hearing objection on this either

    Dom: (but not much excitement either)

    RESOLUTION: Exposing MediaStream in workers is reasonable but
    low priority

     [62]How should MediaStreamTrack interact with BFCache?

      [62] https://github.com/w3c/mediacapture-main/issues/974

    [63][Slide 39]

      [63] 
https://lists.w3.org/Archives/Public/www-archive/2024Feb/att-0008/WEBRTCWG-2024-02-20.pdf#page=39

    Youenn: +1 to this proposal; not just because that's Safari
    current behavior, but also because it will help with getting
    web sites handle device failure situations better
    … would like us to be more proactive on pushing outreach for
    this

    Harald: I've had a lot of discussions on BF-cache in the
    context of peerconnection where we're trying to make WebRTC
    more BF-cache friendly
    … This sounds nice, but I think I'll want to think this through
    some more

    Youenn: this is indeed also worth discussing for WebRTC-PC

    Harald: let's keep discussing this in the issue, want to hear
    from Guido

    Guido: +1

    Youenn: I'll file an issue in webrtc-pc on BF-cache
    friendliness

     [64]Add guidance for defining a new source of MediaStreamTrack

      [64] https://github.com/w3c/mediacapture-main/pull/988

    [65][Slide 40]

      [65] 
https://lists.w3.org/Archives/Public/www-archive/2024Feb/att-0008/WEBRTCWG-2024-02-20.pdf#page=40

    Jan-Ivar: this PR adds clarifications to what muted and ended
    for other sources of tracks

    [66][Slide 41]

      [66] 
https://lists.w3.org/Archives/Public/www-archive/2024Feb/att-0008/WEBRTCWG-2024-02-20.pdf#page=41

    Guido: +1 on the generic guidance; the mic/camera language
    needs more discussion

    Youenn: proposed language sounds good to me; we should review
    our source-defining specs to make sure they're consistent with
    that guidance

    Jan-Ivar: yes, let's file these issues before landing that PR
    … will get that done before the next Editros meeting

    RESOLUTION: Bring [67]#988 to editors meeting for merging after
    having filed issues in source-defining specs

      [67] https://github.com/w3c/mediacapture-main/issues/988

Summary of resolutions

     1. [68]Bring #2935 for merging to editors meeting
     2. [69]Proceed with proposal based on data-minimization
        appraoch
     3. [70]Youenn to revive to Media Capabilities proposal
        media-capabilities#186
     4. [71]Fippo to submit a PR to remove note and a matching WPT
        test
     5. [72]Create extensions repo for screen capture and move
        issues related to new features there
     6. [73]Move forward with a PR to clarify that NotFoundError
        would apply for OS permissions in screen-capture
     7. [74]Make isTypeSupported return a fixed list of answers and
        mark it as deprecated
     8. [75]Supporting captureStream on OffscreenCanvas is
        reasonable but low priority
     9. [76]Exposing MediaStream in workers is reasonable but low
        priority
    10. [77]Bring #988 to editors meeting for merging after having
        filed issues in source-defining specs

Received on Wednesday, 21 February 2024 07:49:08 UTC