Re: Call for Consensus (CfC) on PR 125 (WebRTC Encoded Transform API): "Add API to request key frames"

> On 4 Jan 2022, at 00:58, Bernard Aboba <Bernard.Aboba@microsoft.com> wrote:
> 
> I support merging PR 125. 
> 
> However, I do have a concern about whether the proposed API provides an adequate means of ensuring key synchronization when a new participant joins the conference. 
> 
> When a new participant enters the conference, the sender may desire to generate a key frame and then apply a new encryption key (and keyId) to that keyframe. To ensure synchronization it seems to me that the sender needs to be able to specify precisely when setEncryptionKey() will change the encryption key (and keyId).  
> 
> The updated key needs to be applied to the new keyframe, not to a previous or subsequent frame (which would lead to having the new participant missing video until another keyframe update). 
> 
> If a keyframe is encrypted using the old encryption key, then a newly joined receiver will not be able to decrypt it.  There are a number of ways in which the sender can generate a key frame and then ensure that a key update is applied to that newly generated key frame. The current PR does not make it clear how this would be achieved. 
> 
> One way this might work is to have generateKeyFrame() return a timestamp which is then provided to setEncryptionKey().  It may also be possible to avoid a race condition without passing a timestamp.  


How to handle key frame generation in the case of SFrameTransform needs indeed to be discussed, I filed https://github.com/w3c/webrtc-encoded-transform/issues/127 for that purpose.
The issue with exposing a timestamp to JS is that, by the time JS gets the timestamp, the corresponding frame might already have been encrypted (in case pipeTo is used with a SFrameTransform in particular).
SFrameTransform could have an API that allows defining the key frame strategy on encryption key change.

> 
> From: Bernard Aboba <Bernard.Aboba@microsoft.com>
> Sent: Monday, January 3, 2022 3:21 PM
> To: public-webrtc@W3.org <public-webrtc@w3.org>
> Subject: Call for Consensus (CfC) on PR 125 (WebRTC Encoded Transform API): "Add API to request key frames"
>  
> This is a Call for Consensus (CfC) on Pull Request 125 ("Add API to request key frames") to WebRTC Encoded Transform.  The PR is available for inspection here:
> https://github.com/w3c/webrtc-encoded-transform/pull/125 <https://nam06.safelinks.protection.outlook.com/?url=https%3A%2F%2Fgithub.com%2Fw3c%2Fwebrtc-encoded-transform%2Fpull%2F125&data=04%7C01%7CBernard.Aboba%40microsoft.com%7Ca5d75c8c2cf74f24896308d9cf101405%7C72f988bf86f141af91ab2d7cd011db47%7C0%7C0%7C637768490280501499%7CUnknown%7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D%7C3000&sdata=P8CT%2BjGqmqF9cmrxCCZP80HYS6i5PsfKLdI%2BEhyGh1M%3D&reserved=0>
> 
> A preview of the specification with PR 125 merged is available here: 
> https://pr-preview.s3.amazonaws.com/youennf/webrtc-insertable-streams/pull/125.html <https://nam06.safelinks.protection.outlook.com/?url=https%3A%2F%2Fpr-preview.s3.amazonaws.com%2Fyouennf%2Fwebrtc-insertable-streams%2Fpull%2F125.html&data=04%7C01%7CBernard.Aboba%40microsoft.com%7Ca5d75c8c2cf74f24896308d9cf101405%7C72f988bf86f141af91ab2d7cd011db47%7C0%7C0%7C637768490280501499%7CUnknown%7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D%7C3000&sdata=SBNbKVlaYMPq7LbWaVhmRN20IIkr5e%2FSQnF6QZi8zKY%3D&reserved=0>
> 
> Slides relating to the PR from the December Virtual Interim are available here (starts at slide 17): 
> https://docs.google.com/presentation/d/1jNC69sD-ZeSS7NNHIGcVkOdiZP8m265u6TycEX4G0vw/edit#slide=id.g107baf01ed4_0_97 <https://nam06.safelinks.protection.outlook.com/?url=https%3A%2F%2Fdocs.google.com%2Fpresentation%2Fd%2F1jNC69sD-ZeSS7NNHIGcVkOdiZP8m265u6TycEX4G0vw%2Fedit%23slide%3Did.g107baf01ed4_0_97&data=04%7C01%7CBernard.Aboba%40microsoft.com%7Ca5d75c8c2cf74f24896308d9cf101405%7C72f988bf86f141af91ab2d7cd011db47%7C0%7C0%7C637768490280501499%7CUnknown%7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D%7C3000&sdata=Rw0iLqw36w58quOe24IBtO5t7x2aesus6BmfSs04Dak%3D&reserved=0>
> 
> The CfC will last until Monday January 17, 2022 at midnight Pacific Time.  Please respond with one of the following: 
> 
> I support merging PR 125. 
> I object to merging PR 125, due to reasons included in my comments on the PR (https://github.com/w3c/webrtc-encoded-transform/pull/125 <https://nam06.safelinks.protection.outlook.com/?url=https%3A%2F%2Fgithub.com%2Fw3c%2Fwebrtc-encoded-transform%2Fpull%2F125&data=04%7C01%7CBernard.Aboba%40microsoft.com%7Ca5d75c8c2cf74f24896308d9cf101405%7C72f988bf86f141af91ab2d7cd011db47%7C0%7C0%7C637768490280501499%7CUnknown%7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D%7C3000&sdata=P8CT%2BjGqmqF9cmrxCCZP80HYS6i5PsfKLdI%2BEhyGh1M%3D&reserved=0>)
>  <https://nam06.safelinks.protection.outlook.com/?url=https%3A%2F%2Fgithub.com%2Fw3c%2Fwebrtc-encoded-transform%2Fpull%2F125&data=04%7C01%7CBernard.Aboba%40microsoft.com%7Ca5d75c8c2cf74f24896308d9cf101405%7C72f988bf86f141af91ab2d7cd011db47%7C0%7C0%7C637768490280501499%7CUnknown%7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D%7C3000&sdata=P8CT%2BjGqmqF9cmrxCCZP80HYS6i5PsfKLdI%2BEhyGh1M%3D&reserved=0> 
> Add API to request key frames by youennf · Pull Request #125 · w3c/webrtc-encoded-transform <https://nam06.safelinks.protection.outlook.com/?url=https%3A%2F%2Fgithub.com%2Fw3c%2Fwebrtc-encoded-transform%2Fpull%2F125&data=04%7C01%7CBernard.Aboba%40microsoft.com%7Ca5d75c8c2cf74f24896308d9cf101405%7C72f988bf86f141af91ab2d7cd011db47%7C0%7C0%7C637768490280501499%7CUnknown%7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D%7C3000&sdata=P8CT%2BjGqmqF9cmrxCCZP80HYS6i5PsfKLdI%2BEhyGh1M%3D&reserved=0>
> This includes 3 different APIs: An API to generate a key frame from a RTCRtpScriptTransformer on sender side A corresponding API in RTCRtpSender. This API shares the same algorithm as the first on...
> github.com