- From: Dominique Hazael-Massieux <dom@w3.org>
- Date: Wed, 6 Dec 2023 10:23:57 +0100
- To: public-webrtc@w3.org
Hi,
The minutes of our meeting yesterday (Dec 5) are available at:
https://www.w3.org/2023/12/05-webrtc-minutes.html
and copied as text below.
Dom
WebRTC December 5 2023 meeting
05 December 2023
[2]Agenda. [3]IRC log.
[2] https://www.w3.org/2011/04/webrtc/wiki/December_05_2023
[3] https://www.w3.org/2023/12/05-webrtc-irc
Attendees
Present
Bernard, Carine, Dom, Fippo, Florent, Guido, Harald,
Henrik, Jan-Ivar, PaulAdenot, PeterThatcher, Randell,
Sameer, Shridhar, StefanHolmer, SunShin, TimPanton,
TonyHerre, Youenn
Regrets
-
Chair
Bernard, HTA, Jan-Ivar
Scribe
dom
Contents
1. [4]WebRTC-Encoded Transform: Describe data attribute PR
#212
2. [5]WebRTC Extensions
1. [6]PR #175: Add RTCIceTransport method to remove pairs
3. [7]Issue #2170 Data channel default binaryType value is
'blob'
4. [8]Mediacapture Extensions: latency in Audio Stats
5. [9]WebRTC Extended Use Cases: Low Latency Streaming
1. [10]Low Latency Broadcast w/fanout PR #123
2. [11]Game Streaming
6. [12]RtpTransport
1. [13]Issue #9 Customizing piecemeal
2. [14]Issue #10: SDP Bumper lanes
3. [15]Issue #7: SRTP/cryptex support
4. [16]Issue #13: Workers
5. [17]WHATWG streams
7. [18]Summary of resolutions
Meeting minutes
Slideset: [19]https://lists.w3.org/Archives/Public/www-archive/
2023Dec/att-0000/WEBRTCWG-2023-12-05.pdf
[19]
https://lists.w3.org/Archives/Public/www-archive/2023Dec/att-0000/WEBRTCWG-2023-12-05.pdf
[20]WebRTC-Encoded Transform: Describe data attribute PR #212
[20] https://github.com/w3c/webrtc-encoded-transform/pull/212
[21][Slide 11]
[21]
https://lists.w3.org/Archives/Public/www-archive/2023Dec/att-0000/WEBRTCWG-2023-12-05.pdf#page=11
Fippo: please review so that we can get it merged
jan-ivar: thanks for the PR, it looks good to me - doesn't add
API surface, only describe current one?
fippo: correct
jan-ivar: let's review it in the editors meeting before merging
harald: good to bring attention to this new information during
this meeting
Bernard: this is valuable information, it shouldn't be
normative, but it needs to be somewhere
jan-ivar: anything that stood out as problematic or unusual
when writing this up? e.g. did it reveal concerns with
packetization of certain codecs
hta: the AV1 thing was a surprise
[22]WebRTC Extensions
[22] https://github.com/w3c/webrtc-extensions/
PR [23]#175: Add RTCIceTransport method to remove pairs
[23] https://github.com/w3c/webrtc-extensions/issues/175
[24][Slide 12]
[24]
https://lists.w3.org/Archives/Public/www-archive/2023Dec/att-0000/WEBRTCWG-2023-12-05.pdf#page=12
Sameer: we discussed this in TPAC
Peter: this looks good - 2 minor remarks:
… the array argument makes sense if you 90 candidate pairs to
remove, but it makes it more complicated to figure the return
value
… what's the reason for preventing adding back removed
candidate pairs?
Sameer: you could recreate a pair via remoteAddCandidate(), we
don't have a mechanism for the app to add one
Peter: oh, so the idea is the *browser* doesn't add it back,
makes sense
Bernard: canceling nomination: do you mean remove pair before
nomination?
sameer: relates to the event we added which the app can
preventDefault()
henrik: if performance becomes a concern, it's possible to
batch internally the removals
… the PR LGTM
… no strong feeling re array vs array, but either way can be
optimized
jan-ivar: I don't like the array since I don't think it's
necessary, Promise.all can do the batching
… restartIce() resets things, right?
Sameer: right
jan-ivar: overall this feels a bit like a hack around
nomination because nomination may be too strict
… we could clarify that this is part of the UA-behavior
… I want to make sure we don't overstep anything set in the
IETF spec
Peter: ICE implementations are always free to remove candidate
pairs
Sameer: I'll update the PR in that direction before it
hopefully gets merged then, thanks
[25]Issue #2170 Data channel default binaryType value is 'blob'
[25] https://github.com/w3c/webrtc-pc/issues/2170
[26][Slide 13]
[26]
https://lists.w3.org/Archives/Public/www-archive/2023Dec/att-0000/WEBRTCWG-2023-12-05.pdf#page=13
florent: using Blob to send big files, even if implemented,
wouldn't work as is
Randell: having originally designed this, at this point, I
agree switching the default makes sense
… it's unfortunate the lack of implementation has gained over
the spec
… it is possible to send large files with blob
florent: maxMessageSize doesn't make that practical
Randell: this can be polyfilled on top of datachannel, but this
is water under the bridge
… we should take this as a warning to avoid these incompatible
implementations that last for so long
florent: blob implementation would be possible, but switching
the default would break web-compat
Bernard: the slide should say "Firefox" rather than "Safari"
shipping blob as default
RESOLUTION: change the default for binaryType to arraybuffer
[27]Mediacapture Extensions: latency in Audio Stats
[27] https://github.com/w3c/mediacapture-extensions/pull/134
[28][Slide 14]
[28]
https://lists.w3.org/Archives/Public/www-archive/2023Dec/att-0000/WEBRTCWG-2023-12-05.pdf#page=14
[29][Slide 15]
[29]
https://lists.w3.org/Archives/Public/www-archive/2023Dec/att-0000/WEBRTCWG-2023-12-05.pdf#page=15
Harald: is there a way to figure out when resetLatency was
called?
Henrik: the app knows
Harald: is reset sync?
Henrik: yes
Paul: no objection on my end
RESOLUTION: adopt PR
[30]WebRTC Extended Use Cases: Low Latency Streaming
[30] https://github.com/w3c/webrtc-nv-use-cases/
Low Latency Broadcast w/fanout PR [31]#123
[31] https://github.com/w3c/webrtc-nv-use-cases/issues/123
[32][Slide 19]
[32]
https://lists.w3.org/Archives/Public/www-archive/2023Dec/att-0000/WEBRTCWG-2023-12-05.pdf#page=19
[33][Slide 20]
[33]
https://lists.w3.org/Archives/Public/www-archive/2023Dec/att-0000/WEBRTCWG-2023-12-05.pdf#page=20
Bernard: the use cases in 3.2.2 have very different
requirements
… e.g. very low latency is likely only relevant to sporting
events
… PR [34]#123 attemps to refocus it on very low latency use
cases (auctions)
[34] https://github.com/w3c/webrtc-nv-use-cases/issues/123
[35][Slide 21]
[35]
https://lists.w3.org/Archives/Public/www-archive/2023Dec/att-0000/WEBRTCWG-2023-12-05.pdf#page=21
[36][Slide 22]
[36]
https://lists.w3.org/Archives/Public/www-archive/2023Dec/att-0000/WEBRTCWG-2023-12-05.pdf#page=22
Bernard: ^ proposed new text with PR 123 applied
[37]PR 123 preview of auction use case
[37]
https://pr-preview.s3.amazonaws.com/w3c/webrtc-nv-use-cases/pull/123.html#auction
jan-ivar: could we remove "millions" from the use case?
bernard: sure, more likely relevant in the context of a webinar
jan-ivar: will add to the PR review
[thumbs up from folks on the call with going forward with the
PR]
Game Streaming
[38][Slide 23]
[38]
https://lists.w3.org/Archives/Public/www-archive/2023Dec/att-0000/WEBRTCWG-2023-12-05.pdf#page=23
Bernard: performance is a growing issue, esp in the context of
4K
[39][Slide 24]
[39]
https://lists.w3.org/Archives/Public/www-archive/2023Dec/att-0000/WEBRTCWG-2023-12-05.pdf#page=24
[40][Slide 25]
[40]
https://lists.w3.org/Archives/Public/www-archive/2023Dec/att-0000/WEBRTCWG-2023-12-05.pdf#page=25
bernard: [41]#103 is feedback from youenn - formulating N37 in
terms of API requirements
[41] https://github.com/w3c/webrtc-nv-use-cases/issues/103
[42][Slide 26]
[42]
https://lists.w3.org/Archives/Public/www-archive/2023Dec/att-0000/WEBRTCWG-2023-12-05.pdf#page=26
bernard: PR [43]#125 attempts to do that by referring
explicitly to control of hardware acceleration
… this relates to discussion we had e.g. about events for
hardware failure
[43] https://github.com/w3c/webrtc-nv-use-cases/issues/125
Randell: what does "controlling" mean from a stack perspective?
Bernard: be able to control whether you specifically want
hardware acceleration
… Web Codecs has a preferHardware switch
… We have an issue to raise events when fail over from hardware
to software
Randell: notification seems useful, but that seems different
from this specific requirement
… the requirement here seems to be on implementation
Bernard: no, it's an implementation on the API
… part of the issue is not knowing why the failover happened
doesn't let developers determine whether they can fix the
situation
Randell: the goal seems right, but the wording remains too
fuzzy in terms of requirements
Henrik: "controlling" is the right word - this isn't just
"prefering" for a given codec and leaving it to the UA to
decide
… the app needs to know what's possible and make a decision
based on the available hardware
randell: so exposing more information about current hardware
support for streams that are running and potential hw support
for streams that are offered
Bernard: there is also the big mess around receivable /
sendable codecs
hta: control here is a keyword for "can you do this?", "do this
and let me know if you can't"
jan-ivar: hw control is a requirement today, but it may not be
the case tomorrow
… maybe we can specify the requirements in a way that makes it
clear that it requires hw support today
randell: hw acceleration is mentioned as an example of a
solution in the current language
jan-ivar: there will be concerns in terms of privacy review
bernard: please suggest changes to the PR
jan-ivar: how about "...acceleration if necessary"
henrik: SGTM
bernard: +1
Sun: PR [44]#118 touches on other requirements for game
streaming
[44] https://github.com/w3c/webrtc-nv-use-cases/issues/118
[45][Slide 28]
[45]
https://lists.w3.org/Archives/Public/www-archive/2023Dec/att-0000/WEBRTCWG-2023-12-05.pdf#page=28
Sun: we propose 2 kind of requirements: "fast recovery"
[46][Slide 29]
[46]
https://lists.w3.org/Archives/Public/www-archive/2023Dec/att-0000/WEBRTCWG-2023-12-05.pdf#page=29
Sun: and 2 requirements for "consistent latency"
[47][Slide 30]
[47]
https://lists.w3.org/Archives/Public/www-archive/2023Dec/att-0000/WEBRTCWG-2023-12-05.pdf#page=30
Sun: we discussed this during TPAC
Randell: I support this; when we first designed WebRTC, we
purposefully didn't specify what happened in terms of decoding
on frameloss
… Support for non-full I-frame collections is possible as well
Bernard: is there any API change surfaced to JS needed to
support this?
… I could see this needed in the context of WebCodecs, but not
sure for WebRTC
Sun: at minimum, the ability to set a flag on whether to enable
fast recovery
Bernard: not just an SDP parameter then?
Peter: supporting this would require a large change to
libwebrtc video jitter buffer - is that realistic?
… rtptransport has custom jitter buffer as a use case
Jan-Ivar: I'm unsure W3C needs to do something, vs having this
being negotiated via SDP with user-agent deciding whether to
support it
Randell: it could be done via SDP, but it feels a bit of a hack
… re implementation, having done this in the past, it would
indeed require changes in the jitter buffer, but it doesn't
seem like a major rewrite, more like a small or medium amount
of work to continue decoding while waiting for an i-frame (with
errors in the buffers)
Harald: if it doesn't change the bits on the wire, it shouldn't
be done via SDP
… so in my mind, it does require an API
Henrik: if this was possible to do, would there be any reason
not to have it on all the time? does it add overhead? does it
need a control knob?
Bernard: there are probably applications that wouldn't want do
this, e.g. in the context of a legal case
Dom: maybe we need a use case for the "off" mode before we say
we need it
Sun: please bring more feedback on github
[48][Slide 31]
[48]
https://lists.w3.org/Archives/Public/www-archive/2023Dec/att-0000/WEBRTCWG-2023-12-05.pdf#page=31
Bernard: if this is about how to do it, it probably needs
discussion in IETF given discussions about how well RPSI works
… I'm not sure there is a WebRTC API issue vs implementation or
IETF
… it's most complicated for conferences (compared to the
one-to-one game streaming scenario)
Sun: so I guess we should remove this req based on that
feedback
Bernard: +1 - let's take it up in IETF, it's an important
problem
[49][Slide 32]
[49]
https://lists.w3.org/Archives/Public/www-archive/2023Dec/att-0000/WEBRTCWG-2023-12-05.pdf#page=32
Bernard: which messages are you interested in getting the
interval of? PLI? Receiver reports?
Shridhar: tranport@@@
HTA: can this be uplifted a bit? what you want is to have a
definitive time limit by when you know if a packet was lost
… it might be good better to say that rather than speak of
specific RTCP messages
… this would make a more concrete requirement that allows
different implementations
Sun: we'll update the requirement in that direction
[50][Slide 33]
[50]
https://lists.w3.org/Archives/Public/www-archive/2023Dec/att-0000/WEBRTCWG-2023-12-05.pdf#page=33
Randell: how much of this is talking about the API vs the
implementation?
Sun: no API change required here
Randell: so more of a commentary of the implementation in
libwebrtc
… not really in scope for the WG
Sun: thanks for the feedback
Jan-Ivar: there is an API for jitter buffer target
Randell: that's mostly to have more delay
jan-ivar: or less, in theory
Sun: we'll update the PR based on the discussions
[51]RtpTransport
[51] https://github.com/w3c/webrtc-rtptransport
[52][Slide 37]
[52]
https://lists.w3.org/Archives/Public/www-archive/2023Dec/att-0000/WEBRTCWG-2023-12-05.pdf#page=37
[53]RtpTransport Reminder
[53]
https://docs.google.com/document/d/1cousnS2rmHRH7GEMjWfHTisdUxD2FSVHREbgkbZhuCE/edit#heading=h.2vx6woi9m2uv
[54][Slide 39]
[54]
https://lists.w3.org/Archives/Public/www-archive/2023Dec/att-0000/WEBRTCWG-2023-12-05.pdf#page=39
[55][Slide 40]
[55]
https://lists.w3.org/Archives/Public/www-archive/2023Dec/att-0000/WEBRTCWG-2023-12-05.pdf#page=40
[56][Slide 41]
[56]
https://lists.w3.org/Archives/Public/www-archive/2023Dec/att-0000/WEBRTCWG-2023-12-05.pdf#page=41
[57][Slide 42]
[57]
https://lists.w3.org/Archives/Public/www-archive/2023Dec/att-0000/WEBRTCWG-2023-12-05.pdf#page=42
[58][Slide 44]
[58]
https://lists.w3.org/Archives/Public/www-archive/2023Dec/att-0000/WEBRTCWG-2023-12-05.pdf#page=44
[59][Slide 45]
[59]
https://lists.w3.org/Archives/Public/www-archive/2023Dec/att-0000/WEBRTCWG-2023-12-05.pdf#page=45
Peter: 3 possible general directions I want to review today
[60][Slide 46]
[60]
https://lists.w3.org/Archives/Public/www-archive/2023Dec/att-0000/WEBRTCWG-2023-12-05.pdf#page=46
[61][Slide 47]
[61]
https://lists.w3.org/Archives/Public/www-archive/2023Dec/att-0000/WEBRTCWG-2023-12-05.pdf#page=47
[62][Slide 48]
[62]
https://lists.w3.org/Archives/Public/www-archive/2023Dec/att-0000/WEBRTCWG-2023-12-05.pdf#page=48
[63][Slide 49]
[63]
https://lists.w3.org/Archives/Public/www-archive/2023Dec/att-0000/WEBRTCWG-2023-12-05.pdf#page=49
[64][Slide 50]
[64]
https://lists.w3.org/Archives/Public/www-archive/2023Dec/att-0000/WEBRTCWG-2023-12-05.pdf#page=50
[65][Slide 51]
[65]
https://lists.w3.org/Archives/Public/www-archive/2023Dec/att-0000/WEBRTCWG-2023-12-05.pdf#page=51
Harald: another dimension is whether you can talk to existing
end points
… you can't with the new datachannel - in which case, what's
the advantage over using WebTransport or Quic
… the piecemeal approach doesn't need to be complex for the
simple case - you just let through the things you don't want to
modify
… as long as you stay within certain parameters, you can talk
to people that are not you, which I like
… you can talk with endpoints that talk basic SRTP
… it provides for incremental deployments and incremental
implementations, by focusing on the particular function we care
about today
Peter: you're correct with regard to the new datachannel
speaking to existing end points
… in terms of Quic - they're not P2P in the browser today; we
could fix that
… I agree with your points re piecemeal
Jan-Ivar: in my slides, I'm not proposing an rtp data channel,
I agree with the limitations you and Harald have pointed out
… I think we should only expose RTP as needed (maybe that
points toward piecemeal)
… we need to understand better the scope of the problems we
want to address
Peter: so hearing more support for piecemeal (bit by bit adding
to the existing API, but with likely a low level API)
… I've done some exploration in this space
Bernard: dall-e did a good job representing the data channel
proposal being an isolated island
… I would not favor that
… re piecemeal customization - if you can get the flexibility
without the complexity, this would be ideal
… but we have to demonstrate it
… the incremental deployability is also appealing
Youenn: another vote for piecemeal - a web site should be able
to override some parts of the pipeline, but with the UA taking
over at some point
… complexity is an important consideration
… we've approached this by exposing injection points into the
pipeline, we should continue with that approach
… we should compare with the approach we're exploring in
encoded-transform or to bring web codecs in
Henrik: another benefit of piecemeal is in terms of shipping it
from an implementation perspective; not enough expertise on RTP
to say much more
Issue [66]#9 Customizing piecemeal
[66] https://github.com/w3c/webrtc-rtptransport/issues/9
[67][Slide 53]
[67]
https://lists.w3.org/Archives/Public/www-archive/2023Dec/att-0000/WEBRTCWG-2023-12-05.pdf#page=53
Peter: if we go with piecemeal, this can be done - suggest
following the taxonomy in rfc 7656, most importantly RtpStream
… building on top of that
… will flush this out more
Bernard: some of this would be around custom FEC/custom RTX
scenarios?
Peter: right; I have explored a number of scenarios with that
approach
Stefan: would you insert packets into these RtpStream objects?
Peter: capture what has been sent, allow it through if you're
not customizing, or modifying it and injecting it back
… with lots of implication in terms of congestion control - it
gets complex but I think it's feasible
Harald: once you allow to insert packets then you can generate
packets yourself, in either direction
Peter: absolutely; this is a superset of what I presented to
TPAC in that sense
Jan-Ivar: I'm worried by "full piecemeal customisation" - I
don't think we need to go to the extreme low level of exposing
everything of the RFC on RTP
Peter: to be clear, I'm only asserting something is feasible,
not any API shape
… I'll come back with more specific API proposals, and others
can make proposals as well
Henrik: I get similar vibes to encoded transform from this
slide
… which has proved a powerful pattern
Issue [68]#10: SDP Bumper lanes
[68] https://github.com/w3c/webrtc-rtptransport/issues/10
[69][Slide 54]
[69]
https://lists.w3.org/Archives/Public/www-archive/2023Dec/att-0000/WEBRTCWG-2023-12-05.pdf#page=54
Peter: if we go with piecemeal, I think we'll end up with
lightly enforced bumper lanes
Bernard: Harald raised a concern with avoiding interferences
with the existing peerconnection; this looks to be in the good
direction
Peter: this may depend on whether we allow creating transports
outside of a peerconnection
Harald: I think there should be an API for reserving things,
e.g. to reserve an SSRC
… the API might expose names (e.g. extensions by URIs) rather
than by numbers which avoids the issue of conflicts
Peter: +1; interesting idea on names vs numeric ids
Harald: I'm hitting this with fanout land where we're
discussing using mime types rather than payload types
Issue [70]#7: SRTP/cryptex support
[70] https://github.com/w3c/webrtc-rtptransport/issues/7
[71][Slide 55]
[71]
https://lists.w3.org/Archives/Public/www-archive/2023Dec/att-0000/WEBRTCWG-2023-12-05.pdf#page=55
Peter: with piecemeal, I think we could just rely on the
outcome of the SDP negotiation
Bernard: we have a cryptex API in webrtc-extensions; if the
other side supports cryptex, encrypting everything seems
sufficient granularity, don't see value in per-packet control
Issue [72]#13: Workers
[72] https://github.com/w3c/webrtc-rtptransport/issues/13
[73][Slide 56]
[73]
https://lists.w3.org/Archives/Public/www-archive/2023Dec/att-0000/WEBRTCWG-2023-12-05.pdf#page=56
Peter: if there is a e.g. an RtpStream interface, we would need
to ensure it is transferable - that sounds doable
Harald: SGTM; remember that when you transfer, you transfer one
end of the tube; what's at the other end stays attached to
where it was originally, which may have implication on
implementation
Bernard: we support transferable stream, but would RtpStream be
a WHATWG stream?
… transfering individual objects may not be the right thing;
e.g. for custom NACK needs both dealing with receiving Rtp and
sending RTCP
… you need a combination of read and write streams
… this depends a lot on whether we use WHATWG streams or not
Peter: we need to figure the specific things that people would
want to use in workers
… you're right that RTCP is tricky
Bernard: we also need to figure which workers we need to
support, e.g. audio worklets
Jan-Ivar: time-check?
Peter: ok - we'll need to look at some of the other issues
… re WHATWG streams, I think this can be decided at a later
stage once we have figured out the more fundamental question
WHATWG streams
[74][Slide 63]
[74]
https://lists.w3.org/Archives/Public/www-archive/2023Dec/att-0000/WEBRTCWG-2023-12-05.pdf#page=63
(issue [75]#8)
[75] https://github.com/w3c/webrtc-rtptransport/issues/8
[76][Slide 64]
[76]
https://lists.w3.org/Archives/Public/www-archive/2023Dec/att-0000/WEBRTCWG-2023-12-05.pdf#page=64
[77][Slide 65]
[77]
https://lists.w3.org/Archives/Public/www-archive/2023Dec/att-0000/WEBRTCWG-2023-12-05.pdf#page=65
[78][Slide 66]
[78]
https://lists.w3.org/Archives/Public/www-archive/2023Dec/att-0000/WEBRTCWG-2023-12-05.pdf#page=66
[79][Slide 67]
[79]
https://lists.w3.org/Archives/Public/www-archive/2023Dec/att-0000/WEBRTCWG-2023-12-05.pdf#page=67
Harald: transform has proved a limiting model - a transform is
combining a sending and receiving API together
… we should try to reduce the coupling between components, not
increase it
Bernard: more work needed on this API; we talked about having a
design team to work on this to advance more quickly on this in
the general meetings
… and advance the explainer and specs into more fleshed out
states
… next meeting next week
Summary of resolutions
1. [80]change the default for binaryType to arraybuffer
2. [81]adopt PR
Minutes manually created (not a transcript), formatted by
[82]scribe.perl version 221 (Fri Jul 21 14:01:30 2023 UTC).
Received on Wednesday, 6 December 2023 09:24:01 UTC