Re: [webrtc-encoded-transform] Revisit question of whether frames need to be ordered on incoming (#119)

My sense is that providing unordered frames would provide developers with the most flexibility to develop applications with the lowest possible latency. If I understand correctly, application code will already need to tolerate lost frames, so tolerating out of order frames doesn't seem like it would impose too much more burden.

Offering a configurable buffer as an alternative interface could make things simpler for developers who don't wish to have as much control or don't require such low latency, but that could also be done in Javascript.

Besides the question of the usability and affordances of the interface, are there relevant tradeoffs with regard to overall memory footprint whether a jitter buffer is owned by the browser or by Javascript code?

The one reason I could see this as a "footgun" is that for some (many?) use cases, out of order frames may not occur often enough to be noticeable and therefore developers may mistakenly assume the interface guarantees ordered frames. This could be mitigated by being explicit in the specification and API documentation.

-- 
GitHub Notification of comment by englishm-mux
Please view or discuss this issue at https://github.com/w3c/webrtc-encoded-transform/issues/119#issuecomment-1282607030 using your GitHub account


-- 
Sent via github-notify-ml as configured in https://github.com/w3c/github-notify-ml-config

Received on Tuesday, 18 October 2022 15:45:07 UTC