Re: [webrtc-extensions] Testability of playoutDelayHint (#12)

> I would not expect it to affect the jitter buffer when the hint is decreasing but not when increasing.

Well, _if_ a one second change in jitter buffer depth affected playback, then a one second skip ahead (avoiding audio clicks) might be tolerable in some use cases, whereas a second of frozen silence maybe less so? (speculation)

> It makes you wonder: is the metric broken, or is there additional buffer happening that is not accounted for in the jitter buffer?

This is my question as well. I think we have a spec problem because the implementation appears to pass when it shouldn't:
1. [playoutDelay](https://w3c.github.io/webrtc-extensions/#dom-rtcrtpreceiver-playoutdelay) says explicitly _"The [user agent](https://w3c.github.io/webrtc-extensions/#dfn-user-agent) SHOULD NOT playout audio or video that is received unless this amount of time has passed in seconds"_
    - ✅ PASS (talks only about playout)

3. In parallel: _"If the given delay value is ... above [maximum allowed delay](https://w3c.github.io/webrtc-extensions/#dfn-maximum-allowed-delay) then the value used MUST be clamped"_
    - ✅ PASS (talks about the _"value used"_ below for internal buffering) 

6. _"Modifying the delay of the underlying system SHOULD affect the internal audio or video buffering gradually in order not to hurt user experience. Audio samples or video frames SHOULD be accelerated or decelerated before playout, similarly to how it is done for[ audio/video synchronization](https://www.rfc-editor.org/rfc/rfc5888#section-7) or in response to congestion control."_
    - ✅ PASS (talks about internal buffering "before" playout)

I think the problem is that from a pedantic read there are two independently observable effects: jitter and delay.
- Statement 1 talks only about playout.
- Statement 2 and 3 appear to ultimately only talk about internal (jitter) buffering

If these effects are allowed to be controlled independently, then the implementation passes.

I think this is wrong.

### What is the desired effect?

IMHO the API should only chase one goal effect: jitter-free media. Delay is an undesirable side-effect of that. This means:
1. Statement 1 and 2 seem contradictory (is effect clamped or not?)
2. Statement 1 and 3 seem contradictory (is effect gradual or not?)
3. Any non-jitter-buffer buffering defeats this goal (a wasted opportunity to handle NACKs)

`playoutDelay` may have a terminology problem, e.g. like if my car had a drag-ometer. This is why I opened #156.

-- 
GitHub Notification of comment by jan-ivar
Please view or discuss this issue at https://github.com/w3c/webrtc-extensions/issues/12#issuecomment-1500403683 using your GitHub account


-- 
Sent via github-notify-ml as configured in https://github.com/w3c/github-notify-ml-config

Received on Friday, 7 April 2023 15:47:08 UTC