Re: Proposal-ORTC Sender / Receiver Capabilities Based Model

On Fri, May 9, 2014 at 3:12 PM, Robin Raymond <robin@hookflash.com> wrote:

>
> [RR] see inline:
>
>   Peter Thatcher <pthatcher@google.com>
>  May 9, 2014 at 2:14 PM
> OK, I've had to read through the recent threads and this proposal fairly
> thoroughly.  My thoughts are:
>
> 1.  Most, maybe all, of this, like createParameters and filterParameters
> in the first place, is convenience and could be implemented in a JS
> library.  For such things, we should set a high bar for value vs. cost.
>
> [RR] I wish it could be done as a JS abstraction, but right there's
> insufficient information to be able to write such a JS abstraction layer
> (at the moment). Our intended use for "Capabilities" was only contain to
> contain enough information for another engine to interpret capabilities
> based on an existing understand of codecs and extensions already supported.
> In other words, it can read a parameters object about that codec but it
> does not contain enough information for a JS abstraction to create
> parameters from a JS abstraction layer. It's an important distinction. For
> example, if I were writing a JS abstraction layer that was designed to
> configure SVC layering (e.g. for spatial and temporal), how could I
> enumerate the codecs and know a) which codecs actually support those
> features b) the details of the layering restriction(s) c) other odd
> settings that might be required like the codec defining an SSRC per layer
> or d) if the SVC layering information is contained in-band RTP/RTCP vs
> needs to be manually configured on the receiver (etc, etc.). If this were
> possible to write in a JS abstraction, I would not have proposed this
> createParameters extension.
>
>
​If RtpCapabilities needs to contain more ​info, then we should identify
what info it needs and add it.  We should start there and then worry about
what comes on top.



> 2.  This adds a huge amount of extra complexity (cost) to the API surface.
>  We'd go from having Parameters and Capabilities to having Capabilities,
> Parameters, Preferences, Options, Settings, and Details.  Your first
> advantage listed is "simplicity".  I must humbly disagree.  I think this is
> all very complex.  However, if this is simplicity from your perspective,
> then I have good news: you can implement most (all?) of it in JS as a
> library :).
>
> [RR] I think you'll find you are not correct if I walked you through it
> and you understood why I did what I did. Many things are already proposed
> just repackages a bit cleaner (to me) or made to work. I added "hints"
> called "preferences" to createParameters to make "createParameters" more
> useful. But that part is minimal. I also looked at what was needed for
> audio since audio vs video since the concentration was focused on video
> when it was designed. I went through the details of what was required for
> various forms of SVC. I think you'll find that most everything there on the
> "params" side is merely for completeness and function and not to really
> make it more convenient. I'd be happy to explain one by one why each and
> every item exists to aid the discussion which are needed or not.
>

​I don't think SVC is even a use case that we should worry about much with
createParameters/filterParameters.  ​I mean, it might still function
properly, but I don't think we should worry about it too much.
 createParameters/filterParameters is really meant for simple use cases.
 If you're doing SVC, you're doing multiway video, which is an advanced use
case.  Build the RtpParameters yourself.  It's not that hard.  Or use a
library someone built.



>
> [RR] The only part of the API I'm not convinced is truly needed is
> "simulcasting". I'm still of the belief the browser doesn't need to do any
> coordination work in sending simulcast streams and each sending stream
> could be setup via GUM with separate senders. The only benefit is for a
> receiver being able to wire-up multiple switched simulcast streams to a
> single video output window. I consider this of marginal benefit vs the
> surface it creates and (to me) and could be a prime candidate for JS
> abstraction and removal (but I think others might disagree). Mind you, I
> don't believe the way we currently do 'simulcasting' via "encoding
> parameters defined as layers" will work well in practice (and I have many
> reasons why) that why I made changes with simulcasting in the first place.
>

We debated the multi-RtpReceiver vs. single-RtpReceiver a lot before, and I
thought we agreed on allowing multi-RtpReceiver, and I thought the
arguments for it were pretty compelling.  Or you no longer in agreement
with the decision?​



>
>
> 3.  Most of what you are trying to do seams to be to make it easy to
> create an SVC setup.  In other words, SVC/simulcast without touching
> RtpParameters.  As far as I know, then only use case for SVC/simulcast is
> for multiway video.  Multiway video *is not a simple use case* and I don't
> think we should be adding lots of convenience stuff to the API to cover
> that use case.  We should certainly make it possible to do multiway video
> (provide the power), and certainly JS libraries on top can make it easier
> (provide the convenience), but the convenience functions built into the
> API, such as createParameters and filterParameters, can only cover a
> certain set of simple use cases.  Multiway video is beyond what those can
> provide, unless there's a really simple way of doing it (so that the value
> outweighs the cost).
>
> [RR] No, most is not to make SVC easier to setup. A lot of what I defined
> was by going through use cases and trying to see if they could be made
> would work or not. I also realized a) SVC was not complete in its support
> for various operating modes (eg, temporal) and b) other items were
> incomplete or not defined well and/or could not work for the use cases
> originally described. We could cut some stuff out by saying (for example)
> we only support spatial for now. One of the big advantages of this API and
> conferencing with SVC is a target use case AFAIK. I didn't add a lot of
> "convenience" for the sake of SVC. True, I did add a few knobs to
> createParameters to say "set layering for me please so I don't screw it up"
> but that's it.
>

​Bernard brought up temporal scalability and sort of proposed adding a
field for it, which I think makes sense.  Adding one field is pretty easy
and looks simple.  ​But what you have, no offense, seems like a huge change
and very complex.



>
>
> 4.  Having a simplified "here's the kind of thing I'm looking for" as an
> optional parameter to createParameters might be worth it (more value than
> cost), but I think it would need to be very simple.  I think that might be
> worth exploring in a much more limited capacity.
>
>  [RR] That is a useful exercise to go through an decide what is needed.
>
>
> tl;dr: I see a lot of complexity and not a lot of benefit.   The idea of
> adding an extra parameter to createParameters might be worth it in a much
> more simple version.
>
>  [RR] All I can say is that benefit is subjective. For me, being able to
> setup SVC easily for an SFU scenario is extremely beneficial, especially
> when the Capabilities doesn't tell me which codecs can do what, what
> exactly they need of the "parameters" configuration wise to work, and which
> RTP extensions they require, etc.
>
​
Again, let's make sure the Capabilities give the information the JS needs.
 Then you can do whatever you want with the Parameters.  It sounds like
your needs are specific enough that you shouldn't be using createParameters
in the first place.  ​
​



>
> [RR] Finally, I much appreciate you going through it and I'd be happy to
> explain why I did what I did. Nothing suggested was accidental (I think).
> Perhaps it might be beneficial if I was able to go over the reasoning for
> ever item to help filter (pardon the pun) what we think is truly missing /
> valuable vs what can wait, is wrong, or not truly needed.
>

​Maybe if you broke it down into very small pieces so I can understand it.
 Like, make the diff (or diffs) as small as possible.​  Right now, it hurts
my head.