Simulcast AND layering (was: Handling simulcast)

On 6 September 2013 10:38, Cullen Jennings (fluffy) <fluffy@cisco.com> wrote:
> The point Justin has made before is the solution below won't work for layered codecs and it would be nice to have the JS code be the same and let the browser take care of negotiating if we use simulcast or layered codecs.

I think that scalable codecs are an entirely different problem.  In
that you don't have any option to coerce the API into generating a
reasonable simulacrum of the same.  Simulcast allows that...sort of.

If you want a complete layering+simulcast solution, and I think that
we might ultimately need such a beast, then a set of constraints might
be appropriate.  If you want to go the indirected (i.e., constraints)
path, I'd go for a couple of levels of specificity:

 { "layers": [ "temporal", "spatial" ] }
 { "simulcast": [ "spatial" ] }

Where the labels are:
 temporal= create a layer or simulcast stream that omits every other frame
 spatial= create a layer/simulcast stream that is smaller (by half or
a quarter perhaps)
 quality= create a layer/simulcast stream that has fewer bits (again,
by half or some other factor we decide)
And the list always includes a final layer or simulcast track that has
all the pixels, bits and frames that are in the track.

And maybe something more specific if greater control is desired:
 { "layers": [ { "scalability": "temporal", "frameRate": 15 }, {
"scalability": "quality", "bitrate": 10 } ] }

One consequence of this is that this will manifest on the receiver end
as a single track.  The constituent parts of that track (the lower
layers, the simulcast components) will be inaccessible to the
application.  It will also appear as a track that has the properties
of the highest quality/scale/rate component, even when it doesn't.
Unless the track properties are reflective of the current state, which
could change constantly.  Ideally, there should be some way to look at
the track and learn about this property, so that applications don't
make strange assumptions about the properties of the track.

Received on Friday, 6 September 2013 17:55:58 UTC