Re: [webrtc-extensions] What happens if scaleTo: {360p, 720p} and frame is 360 (#222)

Here's an example of using this API:
```
1. Initially configure {360p, 720p} with scaleTo and a track that is 720p.
2. Make 720p layer active=false. {360p, off} is achieved.
3. No need to do 720p effects processing so the track is changed to 360p.
```

So far so good, no race and we save CPU. But now we want to enable 720p again:
```
1. Increase size of track to 720p.
2. Re-enable the 720p encoding: {360p, 720p}.
```

Both step 1) and step 2) here happen in-parallel.
- The concern is that if an old 360p frame that was emitted by the track prior to step 1) reaches the encoder _after_ the encoding has been re-activated in step 2).

If it is the case that any 720p frame emitted by step 1) always reaches the encoder before any activation instruction from step 2) reaches the encoder then there is no race here. But if media passes by some thread that the setParameters call is not passing through then there could be a race here where "active=true" happens before "720p reaches encoder".

If this race is a non-issue then I agree it's better not to inactivate encodings on behalf of the app since the app should know what it's doing. @jan-ivar 

-- 
GitHub Notification of comment by henbos
Please view or discuss this issue at https://github.com/w3c/webrtc-extensions/issues/222#issuecomment-2346673841 using your GitHub account


-- 
Sent via github-notify-ml as configured in https://github.com/w3c/github-notify-ml-config

Received on Thursday, 12 September 2024 15:57:09 UTC