- From: Dominique Hazael-Massieux <dom@w3.org>
- Date: Wed, 18 Jan 2023 10:53:05 +0000
- To: "public-webrtc@w3.org" <public-webrtc@w3.org>
Le 17/01/2023 à 23:06, Harald Alvestrand a écrit :
> The link is https://youtu.be/aL2-RBuqIq8, and the file is being
> processed as this message goes out.
And the minutes are available at:
https://www.w3.org/2023/01/17-webrtc-minutes.html
And copied as text below.
Dom
WebRTC January 2023 meeting
17 January 2023
[2]Agenda. [3]IRC log.
[2] https://www.w3.org/2011/04/webrtc/wiki/January_17_2023
[3] https://www.w3.org/2023/01/17-webrtc-irc
Attendees
Present
BenWagner, Bernard, Carine, Cullen, Dom, Elad, Florent,
Harald, Henrik, Jan-Ivar, MikeEnglish, PatrickRockhill,
PeterThatcher, TimPanton, TonyHerre, TovePetersson,
Varun, Youenn
Regrets
-
Chair
Bernard, HTA, Jan-Ivar
Scribe
dom
Contents
1. [4]Call for Consensus (CfC) Status
2. [5]WebRTC-pc
1. [6]Issue #2795 Missing URL in RTCIceCandidateInit
2. [7]Issue #2780 duplicate rids in sRD underspecified
3. [8]PR #2801: Prune createAnswer()'s encodings and
[[SendEncodings]] in sLD(answer)
3. [9]WebRTC Extensions
1. [10]Issue #43 / PR #139: Mixed Codec Simulcast
2. [11]Issue #127: How to deal with encoder errors?
3. [12]Issue #130: how does setOfferedRtpHeaderExtensions
work?
4. [13]WebRTC Encoded Transform
5. [14]Screen Capture Community Group
6. [15]Auto-pause Capture
7. [16]MediaStreamTrack Frame Rates
8. [17]Wrap up
9. [18]Summary of resolutions
Meeting minutes
Recording: [19]https://youtu.be/aL2-RBuqIq8
[19] https://youtu.be/aL2-RBuqIq8
IFRAME:
[20]https://www.youtube.com/embed/aL2-RBuqIq8?enablejsapi=1&rel
=0&modestbranding=1
[20]
https://www.youtube.com/embed/aL2-RBuqIq8?enablejsapi=1&rel=0&modestbranding=1
Slideset: [21]https://lists.w3.org/Archives/Public/www-archive/
2023Jan/att-0003/WEBRTCWG-2023-01-17.pdf
[21]
https://lists.w3.org/Archives/Public/www-archive/2023Jan/att-0003/WEBRTCWG-2023-01-17.pdf
Call for Consensus (CfC) Status [22]🎞︎
[22] https://www.youtube.com/watch?v=aL2-RBuqIq8#t=197
[23][Slide 8]
[23]
https://lists.w3.org/Archives/Public/www-archive/2023Jan/att-0003/WEBRTCWG-2023-01-17.pdf#page=8
Harald: we've seen support on low latency use cases - seeing
consensus
… on the Face Detection, there is an objection from Bernard -
we'll have to review it and come back
Youenn: resolving the issues related to low latency use cases
would be needed to declare consensus
Harald: I think we can merge and iterate
Youenn: it's already in the document - I would prefer we remove
the notice "no consensus" once these issues are resolved
Harald: please mark this as an objection on the list then
… More CfC expected
[24]WebRTC-pc [25]🎞︎
[24] https://github.com/w3c/webrtc-pc/
[25] https://www.youtube.com/watch?v=aL2-RBuqIq8#t=364
Issue [26]#2795 Missing URL in RTCIceCandidateInit [27]🎞︎
[26] https://github.com/w3c/webrtc-pc/issues/2795
[27] https://www.youtube.com/watch?v=aL2-RBuqIq8#t=364
[28][Slide 12]
[28]
https://lists.w3.org/Archives/Public/www-archive/2023Jan/att-0003/WEBRTCWG-2023-01-17.pdf#page=12
Youenn: we decided to remove the url from
RTCPeerConnectionIceEvent
… the dictionary to create that event has a candidate field and
an URL field
… that second field should probably be removed
… Usually, events can be shimmed - not for IceEvent, since you
can't create an IceCandidate with an undefined URL
… do we want to change this?
… two questions then: removing URL from IceEventInit, adding
URL to the constructor
Harald: the URL is useful to identify which candidates come
from which servers
… a consturctor that can't create all values is problematic for
testing
… I would like to see that IceCandidate can take a URL to
generate those candidates
Jan-Ivar: no strong opinion; but the fact that the constructor
doesn't have a parameter doesn't prevent it being added to the
object
Youenn: right, but this leaves edge cases where this wouldn't
work as expected
Henrik: no strong opinion, but also finds strange one of these
things can't be constructed
RESOLUTION: mild preference to add url to icecandidate
constructor and consensus to remove url from IceEventInit
Jan-Ivar: would this mean that an JSON-stringified IceCandiate
would include an url?
Youenn: it's a separate issue
Jan-Ivar: they're linked given they use the same dictionary
Youenn: we could define a different dictionary for
json-ification
Jan-Ivar: also needs to consider the impact on addCandidate
Issue [29]#2780 duplicate rids in sRD underspecified [30]🎞︎
[29] https://github.com/w3c/webrtc-pc/issues/2780
[30] https://www.youtube.com/watch?v=aL2-RBuqIq8#t=822
[31][Slide 13]
[31]
https://lists.w3.org/Archives/Public/www-archive/2023Jan/att-0003/WEBRTCWG-2023-01-17.pdf#page=13
[merged since no consensus was expressed on github]
PR [32]#2801: Prune createAnswer()'s encodings and [[SendEncodings]]
in sLD(answer) [33]🎞︎
[32] https://github.com/w3c/webrtc-pc/issues/2801
[33] https://www.youtube.com/watch?v=aL2-RBuqIq8#t=859
[34][Slide 14]
[34]
https://lists.w3.org/Archives/Public/www-archive/2023Jan/att-0003/WEBRTCWG-2023-01-17.pdf#page=14
[merged since no consensus was expressed on github]
[35]WebRTC Extensions [36]🎞︎
[35] https://github.com/w3c/webrtc-extensions
[36] https://www.youtube.com/watch?v=aL2-RBuqIq8#t=904
Issue [37]#43 / PR [38]#139: Mixed Codec Simulcast [39]🎞︎
[37] https://github.com/w3c/webrtc-extensions/issues/43
[38] https://github.com/w3c/webrtc-extensions/issues/139
[39] https://www.youtube.com/watch?v=aL2-RBuqIq8#t=904
[40][Slide 19]
[40]
https://lists.w3.org/Archives/Public/www-archive/2023Jan/att-0003/WEBRTCWG-2023-01-17.pdf#page=19
Bernard: we've discussed this at the July meeting
… Florent developed PR [41]#139
… the use case is for mixed codec simulcast, e.G. you want to
use AV1 but will only get decent performant at low resolution
… you would use a different codec at a higher res (e.g. vp8 or
vp9)
[41] https://github.com/w3c/webrtc-extensions/issues/139
[42][Slide 20]
[42]
https://lists.w3.org/Archives/Public/www-archive/2023Jan/att-0003/WEBRTCWG-2023-01-17.pdf#page=20
[43][Slide 21]
[43]
https://lists.w3.org/Archives/Public/www-archive/2023Jan/att-0003/WEBRTCWG-2023-01-17.pdf#page=21
Bernard: this example puts 2 codecs in - AV1 and VP8; at full
res, only VP8
[44][Slide 22]
[44]
https://lists.w3.org/Archives/Public/www-archive/2023Jan/att-0003/WEBRTCWG-2023-01-17.pdf#page=22
Florent: issue [45]#126 addresses another problem that this PR
would also help cover
… some applications want to select which codec is used but
without renegotiation reordering
… the current approach is heavy, annoying and issue-prone
[45] https://github.com/w3c/webrtc-extensions/issues/126
[46][Slide 23]
[46]
https://lists.w3.org/Archives/Public/www-archive/2023Jan/att-0003/WEBRTCWG-2023-01-17.pdf#page=23
[47][Slide 24]
[47]
https://lists.w3.org/Archives/Public/www-archive/2023Jan/att-0003/WEBRTCWG-2023-01-17.pdf#page=24
Bernard: questions about weird cases in the field
… imagine there is a hardware encoder but is not available
because it gets preempted
… are there situations where having an array would allow for a
better fallback? although this wouldn't help in this case
Florent: wrt limited capacity of hardware encoder- if there is
no software fallback, setParameters would throw if resources
can't be acquired
… although not if that happens later - Henrik has a proposal
for that
… if you run out of capacity on the hardware encoder, there is
no control to surface errors upon software fallback
Bernard: maybe Henrik's proposal will help there indeed
Harald: the renegotiation problem is prettily easily solved:
when you set the encoding, it must be valid; when you negotiate
(even 1st one), you remove anything that isn't in the
negotiated codecs
… for ease of use, we should have an array and use the 1st
entry of the array that are still available after negotiations
Henrik: I have a proposal that somewhat overlaps with it that
we will talk about later
… I do have a preference for a single codec value in
setParameters
… there should either be sensible defaults or have the stream
disabled
… I would keep this API surface as simple as possible
Florent: if the selected codec doesn't match, we could throw an
error for the app to handle
jan-ivar: in the API so far, we've tried hard to keep
setParameters and negotiation deal with the same settings to
avoid creating races
… that may be solved by what Harald described
Florent: the negotation is about what codecs are allowed, not
the ones that are used - the usage is not the same
… at the moment, renegotiation is used to push the first in the
list to get it used, but I think more control is needed
Bernard: there may be a difference between before and after
offer/answer
… after offer/answer, the codecs in the list are within the
negotiated envelop, you check against that, not capabilities
… for addTransceiver, potentially there hasn't been an O/A yet
… if you haven't called setCodecPreferences, it could be any in
capabilities; this could lead to a contradiction with the
addTransceiver
… this has to be thought through, probably iterating in the PR
Florent: we should indeed check against capabilities, codec
preferences; we should align with what is done e.g. in SVC
… maybe sRD should throw an error; developer tools may help
provide more visibility on what SDP would send
… maybe we can iterate on this on github as we prepare the PR
Issue [48]#127: How to deal with encoder errors? [49]🎞︎
[48] https://github.com/w3c/webrtc-extensions/issues/127
[49] https://www.youtube.com/watch?v=aL2-RBuqIq8#t=2100
[50][Slide 25]
[50]
https://lists.w3.org/Archives/Public/www-archive/2023Jan/att-0003/WEBRTCWG-2023-01-17.pdf#page=25
Henrik: somewhat related but also different
[51][Slide 26]
[51]
https://lists.w3.org/Archives/Public/www-archive/2023Jan/att-0003/WEBRTCWG-2023-01-17.pdf#page=26
Bernard: should it always be active=false on all layers? e.g.
if only a given encoder is a source of errors
Henrik: you may want to know which layers the errors happened;
the event may need to surface which encoders this error
occurred
jan-ivar: clarification that this is an event, not callbacks
… is it necessary to set active to false and let JS deal with
the situation overall?
henrik: if the encoder doesn't work, it can't keep encoding: it
needs to be stopped to fallback to a sensible default
… I'm concerned that any default would end up sending
unexpected keyframes
youenn: what might constitute an error? e.G. transient vs fatal
error? this may lead to fragmentation
… do we want to articulate this on error vs not error, or a
change more generally?
henrik: very good point; some errors may simply be a
notification but the app may not need to act because it can be
recovered from
… would be useful to say whether this should include the
fallback in case of codec removal from negotiation
harald: we shouldn't stop anything unless the error forces it
… so setting active=false should only impact affected layers
… it should be an event, since events can have a default
behavior that can be disabled
… so the event could be fired every time there is a significant
change, and by default let it managed by the UA that the app
can intercept
Bernard: +1 to Youenn and Harald
… but I don't think it's for recovery - this is just for real
errors?
Henrik: right, but there may still be fallbacks (hardware →
software)
bernard: if the error is recoverable, I would assume you
wouldn't have that event
Youenn: some OSes are changing from hardware to software based
on the resolution of the stream - not even an error
henrik: maybe this should be scoped to unrecoverable errors?
timp: I like this, but I don't think it's about errors, it
should be codecavailabilitychange
henrik: good point
harald: +1
Issue [52]#130: how does setOfferedRtpHeaderExtensions work?
[53]🎞︎
[52] https://github.com/w3c/webrtc-extensions/issues/130
[53] https://www.youtube.com/watch?v=aL2-RBuqIq8#t=2827
[54][Slide 27]
[54]
https://lists.w3.org/Archives/Public/www-archive/2023Jan/att-0003/WEBRTCWG-2023-01-17.pdf#page=27
Harald: Fippo and I were disagreeing on the interpretation of
the spec - I'm now thinking Fippo is right, but want to make
sure the WG is also comfortable with that interpretation -
please chime in in the issue
jan-ivar: +1 to fippo's interpretation, which is also more Web
compatible
… also frozenarrays in dictionary is frowned upon
youenn: +1 To fippo's as well
[55]WebRTC Encoded Transform [56]🎞︎
[55] https://github.com/w3c/webrtc-encoded-transform
[56] https://www.youtube.com/watch?v=aL2-RBuqIq8#t=2983
[57][Slide 30]
[57]
https://lists.w3.org/Archives/Public/www-archive/2023Jan/att-0003/WEBRTCWG-2023-01-17.pdf#page=30
[58][Slide 31]
[58]
https://lists.w3.org/Archives/Public/www-archive/2023Jan/att-0003/WEBRTCWG-2023-01-17.pdf#page=31
[59][Slide 32]
[59]
https://lists.w3.org/Archives/Public/www-archive/2023Jan/att-0003/WEBRTCWG-2023-01-17.pdf#page=32
[60][Slide 33]
[60]
https://lists.w3.org/Archives/Public/www-archive/2023Jan/att-0003/WEBRTCWG-2023-01-17.pdf#page=33
Harald: I designed an API for frame handling
… creating frames from data and metadata
… modify a frame metadata (in particular to avoid data copy)
… data modification would happen async from metadata - which
raises the question about consistency of the frame
[61][Slide 34]
[61]
https://lists.w3.org/Archives/Public/www-archive/2023Jan/att-0003/WEBRTCWG-2023-01-17.pdf#page=34
Harald: we've been reasonably successful using streams for
frames; but reconfiguration requests are more events-like
[62][Slide 35]
[62]
https://lists.w3.org/Archives/Public/www-archive/2023Jan/att-0003/WEBRTCWG-2023-01-17.pdf#page=35
Harald: I propose an interface to handle this, as previously
presented after the IETF hackathon
[63][Slide 36]
[63]
https://lists.w3.org/Archives/Public/www-archive/2023Jan/att-0003/WEBRTCWG-2023-01-17.pdf#page=36
[64][Slide 37]
[64]
https://lists.w3.org/Archives/Public/www-archive/2023Jan/att-0003/WEBRTCWG-2023-01-17.pdf#page=37
Harald: the long term plan would be to redefine RTPSender /
RTPReceiver as composed of smaller components (encoder,
packetizer)
[65][Slide 38]
[65]
https://lists.w3.org/Archives/Public/www-archive/2023Jan/att-0003/WEBRTCWG-2023-01-17.pdf#page=38
Bernard: is there an assumption that the packetizer is the one
in the browser, or would it be possible to bring your own
packetizer? e.g. would be useful for HEVC in WebRTC
Harald: there is a limited number of behaviors for packetizers
- we should enable these different behaviors, haven't looked at
bringing a fully custom packetizers
Bernard: this may impact the discussion of the use case
… another question about workers: in WebCodec, encode/decode
would typically happen in a worker
… would this imply bringing RTCSender/Receiver in workers?
Harald: unsure about that one
… events aren't transferred
… making objections transferable can prove tricky, as we've
learned with MediaStreamTrack
Peter: +1 to considering these use cases in scope and calling
for proposals
… I would like to get clarity on whether custom packetizer is
part of that though, e.g. for custom SVC
Harald: none of the use cases in my list require custom
packetization, so such use cases would need to be added
… I'm hesitant and somewhat nervous to expose packet levels to
JS, esp without strong supporting use cases
Bernard: what should be the next steps?
Harald: run a CfC on use cases? if approved, then we would
iterate on proposals
Jan-Ivar: the use cases could use a bit more specificity; I'm
worried about having too many APIs to achieve the same thing
… there is already a way to do relay where implementations
could optimize decode/encode
… although the modification use case is a good illustration of
what additional would be needed
… I don't see a problem with using streams in events for the
control path
Youenn: +1 to the question wrt packet vs frame
… if we want to a packet level API, we'll need to figure out
the security model, which will lead to a very different path
Harald: I haven't yet seen the use case written up that
warrants a packet-level API; I'm very happy to see it, discuss
it and decide based on it
… but at the moment, what I've seen needed is possible to do at
the frame level
… hence why I'm pursuing it
… so let's see the use cases
Dom: do we want to wait for these additional use cases before
CfC these ones?
Harald: no, I think they can live on their own; not clear that
the packet level API would fully address them in any case
[66]Screen Capture Community Group [67]🎞︎
[66] https://www.w3.org/community/sccg/
[67] https://www.youtube.com/watch?v=aL2-RBuqIq8#t=4180
[68][Slide 41]
[68]
https://lists.w3.org/Archives/Public/www-archive/2023Jan/att-0003/WEBRTCWG-2023-01-17.pdf#page=41
Toipc: [69]De-adopting Capture Handle Identity
[69] https://github.com/w3c/mediacapture-handle
[70][Slide 44]
[70]
https://lists.w3.org/Archives/Public/www-archive/2023Jan/att-0003/WEBRTCWG-2023-01-17.pdf#page=44
[71][Slide 45]
[71]
https://lists.w3.org/Archives/Public/www-archive/2023Jan/att-0003/WEBRTCWG-2023-01-17.pdf#page=45
Jan-Ivar: the WebRTC WG has been in charge of APIs that produce
or consume MediaSTreamTrack
… I don't think it would be progress from moving specs from W3C
WGs to a CG - it feels like a step backwards
… it's been less than a year since we adopted the spec
… Capture Handle actions is a supplement to identity, not an
alternative
… traditionally, we "de-adopt" a spec due to lack of interest;
Mozilla is definitely interested in this API, so we don't think
it should be de-adopted
Bernard: procedurally, are you suggesting a CfC to de-adopt
Capture Handle?
Elad: what I want to ahppen is Capture Handle Idendity to be be
incubated before being brought back
… I think the Screen Capture CG would be the right place - it
could be either by delegation or copy
Bernard: this would be limited to Capture Handle Identity?
Elad: correct
Youenn: I don't think the WebRTC WG approval is needed to fork
the spec; the CG can do it on its own
… I don't see value in removing it from the WebRTC WG
Jan-Ivar: with regard to disagreements, my view is that they've
been minor - there is overall agreement on the direction
Dom: my preference would be to keep it in the WG since I think
the disagreements are not critical
… but I think a situation where the specs exist in two places
is the worse situation for the community in terms of clarity
RESOLUTION: Start a CfC on de-adopting Capture Handle Identity
[72]Auto-pause Capture [73]🎞︎
[72] https://github.com/w3c/mediacapture-screen-share/issues/255
[73] https://www.youtube.com/watch?v=aL2-RBuqIq8#t=4968
[74][Slide 47]
[74]
https://lists.w3.org/Archives/Public/www-archive/2023Jan/att-0003/WEBRTCWG-2023-01-17.pdf#page=47
[75][Slide 48]
[75]
https://lists.w3.org/Archives/Public/www-archive/2023Jan/att-0003/WEBRTCWG-2023-01-17.pdf#page=48
[76][Slide 49]
[76]
https://lists.w3.org/Archives/Public/www-archive/2023Jan/att-0003/WEBRTCWG-2023-01-17.pdf#page=49
[77][Slide 50]
[77]
https://lists.w3.org/Archives/Public/www-archive/2023Jan/att-0003/WEBRTCWG-2023-01-17.pdf#page=50
[78][Slide 51]
[78]
https://lists.w3.org/Archives/Public/www-archive/2023Jan/att-0003/WEBRTCWG-2023-01-17.pdf#page=51
[79][Slide 52]
[79]
https://lists.w3.org/Archives/Public/www-archive/2023Jan/att-0003/WEBRTCWG-2023-01-17.pdf#page=52
[80][Slide 53]
[80]
https://lists.w3.org/Archives/Public/www-archive/2023Jan/att-0003/WEBRTCWG-2023-01-17.pdf#page=53
[81][Slide 54]
[81]
https://lists.w3.org/Archives/Public/www-archive/2023Jan/att-0003/WEBRTCWG-2023-01-17.pdf#page=54
Youenn: the use case makes sense and is worth saving
… I think the API should be the level at the source, not the
track
… so probably in the capturecontroller
… but we can dive into the API shape once we agree on solving
the issue
TimP: does this cover audio as well? the reasons for pausing
seem video orientated
elad: interesting question; it could support Youenn's point
about capturecontroller
… or maybe we need a source object as I've been discussing with
Ben
… it may be worth having separate control for audio & video who
are perceived differently
TimP: this could be used in case your webrtc call gets mic
stolen e.g. by a GSM call
elad: maybe that's already covered by the muted event, would
need to look at it
TimP: let's look at audio in general
Harald: This use case is definitely worth solving
… traditionally we haven't exposed Sources to JS, which is my
worry everytime we talk about them
… we might want to; maybe the CaptureController is the source
… I would like to mention again preventDefault() in the event
interface that allows to intercept the event default impact
Jan-Ivar: +1 to Youenn on bringing this to capturecontroller
(which is indistinguishable from a source in a case of a single
capture)
… I don't think we should terminate output; events should be
optional, default shouldn't terminate output
… if this doesn't move to capturecontroller, I have other
issues (e.g. confusion between muted/unmuted, paused)
Elad: wrt CaptureController, it makes some sense; but I have
worries about transferability given that CaptureController
isn't transferable
… needs to be evaluated more
… With respect to default actions, a core component of the
proposal is to preserve the legacy behavior - if an event
handler isn't set, the output isn't paused
jan-ivar: setting an eventhaldner shouldn't have a side effect
though
Elad: slide 53 has a possible approach to this
Harald: preventDefault will help with that
Elad: sold
… So next steps include evaluating MediaSTreamTrack vs
CaptureController vs a new Source object?
Harald: +1
[82]MediaStreamTrack Frame Rates [83]🎞︎
[82] https://github.com/w3c/mediacapture-extensions/pull/77
[83] https://www.youtube.com/watch?v=aL2-RBuqIq8#t=5959
[84][Slide 57]
[84]
https://lists.w3.org/Archives/Public/www-archive/2023Jan/att-0003/WEBRTCWG-2023-01-17.pdf#page=57
[85][Slide 58]
[85]
https://lists.w3.org/Archives/Public/www-archive/2023Jan/att-0003/WEBRTCWG-2023-01-17.pdf#page=58
TimP: uncomfortable with "decimated" which should relate to a
factor of 10, not meant here
… what do we think the developer would do with this
information?
Henrik: you can measure deltas between cameras settings and
what you're getting
… you may reconfigure the camera
… also useful for debugging - frames being dropped from camera
issues or other issues
… right now, hard to make sense of frames dropped
TimP: so mostly a diagnostic tool
henrik: yes
jan-ivar: what happens if track.enabled = false?
henrik: that needs to be decided - maybe stop incrementing
counters
youenn: I understand delivered, decimated; is the total sum of
frames those generated by the cameras?
… if so, maybe instead of "dropped", we provide the total as
"framesGenerated"?
henrik: fine with me
jan-ivar: what happens in low-light conditions?
henrik: framesGenerated would lower (in this new model)
henrik: hearing overall support with some proposed changes
Wrap up [86]🎞︎
[86] https://www.youtube.com/watch?v=aL2-RBuqIq8#t=6440
Bernard: a number of action items:
… - CfC on Harald's use cases
… - CfC on de-adoption of capture handle
Elad: please chime in on the issue for auto pause to help with
the next iteration
Youenn: an event on capturecontroller should suffice
Elad: maybe more is needed to distinguish audio/video
… I'll flesh this out
Summary of resolutions
1. [87]mild preference to add url to icecandidate constructor
and consensus to remove url from IceEventInit
2. [88]Start a CfC on de-adopting Capture Handle Identity
Received on Wednesday, 18 January 2023 10:53:10 UTC