Re: Proposal-ORTC Sender / Receiver Capabilities Based Model

[RR] see inline:

> Peter Thatcher <mailto:pthatcher@google.com>
> May 9, 2014 at 2:14 PM
> OK, I've had to read through the recent threads and this proposal 
> fairly thoroughly.  My thoughts are:
>
> 1.  Most, maybe all, of this, like createParameters and 
> filterParameters in the first place, is convenience and could be 
> implemented in a JS library.  For such things, we should set a high 
> bar for value vs. cost.
[RR] I wish it could be done as a JS abstraction, but right there's 
insufficient information to be able to write such a JS abstraction layer 
(at the moment). Our intended use for "Capabilities" was only contain to 
contain enough information for another engine to interpret capabilities 
based on an existing understand of codecs and extensions already 
supported. In other words, it can read a parameters object about that 
codec but it does not contain enough information for a JS abstraction to 
create parameters from a JS abstraction layer. It's an important 
distinction. For example, if I were writing a JS abstraction layer that 
was designed to configure SVC layering (e.g. for spatial and temporal), 
how could I enumerate the codecs and know a) which codecs actually 
support those features b) the details of the layering restriction(s) c) 
other odd settings that might be required like the codec defining an 
SSRC per layer or d) if the SVC layering information is contained 
in-band RTP/RTCP vs needs to be manually configured on the receiver 
(etc, etc.). If this were possible to write in a JS abstraction, I would 
not have proposed this createParameters extension.
>
> 2.  This adds a huge amount of extra complexity (cost) to the API 
> surface.  We'd go from having Parameters and Capabilities to 
> having Capabilities, Parameters, Preferences, Options, Settings, and 
> Details.  Your first advantage listed is "simplicity".  I must humbly 
> disagree.  I think this is all very complex.  However, if this is 
> simplicity from your perspective, then I have good news: you can 
> implement most (all?) of it in JS as a library :).
[RR] I think you'll find you are not correct if I walked you through it 
and you understood why I did what I did. Many things are already 
proposed just repackages a bit cleaner (to me) or made to work. I added 
"hints" called "preferences" to createParameters to make 
"createParameters" more useful. But that part is minimal. I also looked 
at what was needed for audio since audio vs video since the 
concentration was focused on video when it was designed. I went through 
the details of what was required for various forms of SVC. I think 
you'll find that most everything there on the "params" side is merely 
for completeness and function and not to really make it more convenient. 
I'd be happy to explain one by one why each and every item exists to aid 
the discussion which are needed or not.

[RR] The only part of the API I'm not convinced is truly needed is 
"simulcasting". I'm still of the belief the browser doesn't need to do 
any coordination work in sending simulcast streams and each sending 
stream could be setup via GUM with separate senders. The only benefit is 
for a receiver being able to wire-up multiple switched simulcast streams 
to a single video output window. I consider this of marginal benefit vs 
the surface it creates and (to me) and could be a prime candidate for JS 
abstraction and removal (but I think others might disagree). Mind you, I 
don't believe the way we currently do 'simulcasting' via "encoding 
parameters defined as layers" will work well in practice (and I have 
many reasons why) that why I made changes with simulcasting in the first 
place.
>
> 3.  Most of what you are trying to do seams to be to make it easy to 
> create an SVC setup.  In other words, SVC/simulcast without touching 
> RtpParameters.  As far as I know, then only use case for SVC/simulcast 
> is for multiway video.  Multiway video *is not a simple use case* and 
> I don't think we should be adding lots of convenience stuff to the API 
> to cover that use case.  We should certainly make it possible to do 
> multiway video (provide the power), and certainly JS libraries on top 
> can make it easier (provide the convenience), but the convenience 
> functions built into the API, such as createParameters and 
> filterParameters, can only cover a certain set of simple use cases. 
>  Multiway video is beyond what those can provide, unless there's a 
> really simple way of doing it (so that the value outweighs the cost).
[RR] No, most is not to make SVC easier to setup. A lot of what I 
defined was by going through use cases and trying to see if they could 
be made would work or not. I also realized a) SVC was not complete in 
its support for various operating modes (eg, temporal) and b) other 
items were incomplete or not defined well and/or could not work for the 
use cases originally described. We could cut some stuff out by saying 
(for example) we only support spatial for now. One of the big advantages 
of this API and conferencing with SVC is a target use case AFAIK. I 
didn't add a lot of "convenience" for the sake of SVC. True, I did add a 
few knobs to createParameters to say "set layering for me please so I 
don't screw it up" but that's it.
>
> 4.  Having a simplified "here's the kind of thing I'm looking for" as 
> an optional parameter to createParameters might be worth it (more 
> value than cost), but I think it would need to be very simple.  I 
> think that might be worth exploring in a much more limited capacity.
>
[RR] That is a useful exercise to go through an decide what is needed.
>
> tl;dr: I see a lot of complexity and not a lot of benefit.   The idea 
> of adding an extra parameter to createParameters might be worth it in 
> a much more simple version.
>
[RR] All I can say is that benefit is subjective. For me, being able to 
setup SVC easily for an SFU scenario is extremely beneficial, especially 
when the Capabilities doesn't tell me which codecs can do what, what 
exactly they need of the "parameters" configuration wise to work, and 
which RTP extensions they require, etc.

[RR] Finally, I much appreciate you going through it and I'd be happy to 
explain why I did what I did. Nothing suggested was accidental (I 
think). Perhaps it might be beneficial if I was able to go over the 
reasoning for ever item to help filter (pardon the pun) what we think is 
truly missing / valuable vs what can wait, is wrong, or not truly needed.