Multiple codecs for video conferencing [was: RE: <device> proposal (for video conferencing, etc)]

Ian Hickson wrote:
> Assuming we want to ensure that any client can communicate with any other
> client, at a minimum we would need at least one common codec. If we have
> one common codec, then there's no need for others. Thus, I don't see that
> we'd need format negotiation.

If all we care about is whether one client can communicate with another then yes, all we need is one codec that everyone can agree to implement. 

However I think that there are other reasons why, even if everyone could agree to implement one codec, we would still end up wanting to support multiple codecs. 

In particular:

* Hardware support: Some codecs may be supported in hardware. Even if my browser has software support for Theora, I'm still going to strongly prefer H264 if I have hardware decode for it.

* Compression ratio: Some codecs compress better than other codecs. In the future we are likely to see codecs that compress significantly better than the codecs that are currently popular.

* Video type: Some codecs might be specially designed for videoconferencing (e.g. fancy codecs that build up a model of the user's face), but not so great for movies.

* New codecs: People will release better codecs in the future. We don't want to force everyone to stick with H264/Theora/whatever if something better has come along.

* Special features: E.g. codecs that include information for gaze correction, 3D, etc

It would definitely be useful to have a single baseline codec that everyone agreed to implement, however I think that, even if we had that, we would still need to support multiple codecs, and allow clients to negotiate to choose the one that best suited their collective needs.

There are also some scenarios where it makes sense to use several formats in the same conference. E.g. imagine that you and I are videoconferencing in super-HD using our 50-inch monitors, and are then joined by a friend on his phone. Even if the phone supported our super-HD codec, he wouldn't be able to keep up with the data. Either the video would need to be transcoded on the server, or we would need to encode our video at multiple resolutions so that the guy on the phone could get a low-bitrate feed while we still kept our high-bitrate feeds.

Is this case we would need to do some kind of format negotiation between the different parties, even if we were all using the same codec.


Received on Friday, 18 December 2009 01:53:04 UTC