Re: Proposed Resolution: Removal of the createParameter and filterParameter functions

Most but not all existing H.264/SVC implementations do single SSRC, but not necessarily in an identical way. The UCI Forum has a transport spec that describes what Vidyo does but I have no idea whether anyone plans to use Open H.264 (which supports temporal scalability in the encoder) to implement that. So some additional homework might be useful.

Just realized that In addition to the capability question there is also the parameter issue. Assuming we fix the parameter retrieval issue we can know what SSRCs the system chose, but currently we only know what the transport choices were after the fact.

On Jun 16, 2014, at 7:25 PM, "Peter Thatcher" <pthatcher@google.com<mailto:pthatcher@google.com>> wrote:

If implementations can do one or the other but not both, I think an enum would make the most sense.

But would it be possible to just choose one and make that mandatory?  It would make like more simple.




On Mon, Jun 16, 2014 at 4:06 PM, Bernard Aboba <Bernard.Aboba@microsoft.com<mailto:Bernard.Aboba@microsoft.com>> wrote:
Yes, meant intersection. Union would be bad ;)

The issue underlying the multiple SSRC/single SSRC capability is that some codec implementations may support multiple SSRCs with SVC while others do not. For example, some H.264/SVC implementations only support one SSRC, some only support multiple SSRCs. I do not know any that support both, but it is possible. For VP8 this is not an issue, as I understand it - only a single SSRC is supported for temporal scaling.

On Jun 16, 2014, at 5:27 PM, "Peter Thatcher" <pthatcher@google.com<mailto:pthatcher@google.com>> wrote:

I think you meant the intersection of the Sender and Receiver object capabilities, not the union.

Assuming you meant intersection, I like all of the changes you are proposing to RtpCapabilities.  The only one I don't like is  multiStreamSupport.  I don't quite understand the need, and I really don't like the name.  Assuming I understood the need, I think a better name might be something "layeringWithMultipleSsrcs".  But what if a codec supports multiple ssrcs and not single-ssrc?  Do we need to have some kind of sequence of layering types, where it could be multiple-ssrc, single-ssrc, or both?

​Also, it seems unrelated, but you added maxSpatialLayers, maxTemporalLayers, etc.  I like them, too.


On Fri, Jun 13, 2014 at 12:09 PM, Bernard Aboba <Bernard.Aboba@microsoft.com<mailto:Bernard.Aboba@microsoft.com>> wrote:
At the last ORTC Community Group meeting, there was a consensus to remove the createParameter and filterParameter
methods from the ORTC API.  The concern was that, because the operation of these functions was not well defined,
that implementations could differ significantly enough to cause interoperability problems.

However, since the createParameter and filterParameter methods were used within a number of examples within the
specification, once the methods were removed, we needed to figure out how to rewrite the examples.

In particular, Examples 8 and 9 in the Editor’s draft illustrated how to set up a peer-to-peer call involving a single audio
and video stream in only a few lines of code, utilizing single-round trip signaling.

For the removal of createParameters and filterParameters to be credible, we needed to be able to provide examples
of comparable simplicity – but with a better likelihood of interoperation between browsers.   In practice, that meant
sticking to a single round-trip for signaling and not adding significantly to the size of the code, while trying to keep
hand waving to a minimum.   Given how it turned out that there were subtle “gotchas” in createParameters and
filterParameters,    we were somewhat wary of depending on hypothetical functions that were presumed to be
codeable, as opposed to  “running code” that ORTC CG members could play with and verify themselves.
While we have not  gotten all the way to “running code” yet,  we do believe that the exercise of providing it
is important – and whether that “running code” is actually included in the ORTC API specification
or just provided as a link should be discussed.

As a first step, we set out to re-write the examples by exchanging receiver and sender
RTCRtpCapabilities objects, as obtained from the getCapabilities method.   To determine the RTCRtpParameters,
we would compute the union of the Sender and Receiver object capabilities,  using a Javascript library function.
To make sure we weren’t merely replacing the problematic createParameters and filterParameters methods
with hypothetical methods that couldn’t be written (or that would be very complex if reduced to Javascript),
we wrote down the steps that our desired methods would need to go through, and
then examined whether these steps could actually be carried out with a minimum of fuss based on the
RTCRtpCapabilities and RTCRtpParameter objects defined in the editor’s draft.

Here is the prototype of the functions we set out to investigate:

RTCRtpParameters function myCapsToSendParams (RTCRtpCapabilities sendCaps, RTCRtpCapabilities remoteRecvCaps) {
// Method returning the sender RTCRtpParameters, based on the local sender and remote receiver capabilities.
// The goal is to enable a single stream audio and video call with minimum fuss.

RTCRtpParameters function myCapsToRecvParams (RTCRtpCapabilities recvCaps, RTCRtpCapabilities remoteSendCaps) {
// Method returning the receiver RTCRtpParameters, based on the local receiver and remote sender capabilities.
return myCapsToSendParams(remoteSendCaps, recvCaps);
}

Here are the steps we believe the methods would go through:

1. Determine the codecs that the sender and receiver have in common.
2. Within each common codec, determine the union of supported parameters, header extensions and rtcpFeedback mechanisms, and configure them.
3. For each common codec, determine the payloadType to be used, based on the receiver preferredPayloadType.
4. Set RTCRtcpParameters such as rtcp.compound and rtcp.mux to their default values.
5. Return RTCRtpParameters enabling the jointly supported features and codecs.

However, when we looked at whether we could actually execute these steps based solely on the
information in RTCRtpCapabilities defined in the Editor’s draft,  we discovered a few deficiencies.

One missing item was that there was no preferred Payload Type information for each codec in
RTCRtpCapabilities.   Another problem was that the RTCP Feedback capabilities only applied at a
global level within RTCRtpCapabilities, so that we couldn’t figure out which RTCP feedback message
to enable within each codec.  A similar issue was found with Header Extensions - just knowing that
the RTP stack could support a given header extension was not enough to figure out which codecs it
could be configured with.  Also, to fill in the RTCRtpParameters, it was necessary to know what the
preferred IDs were for the header extensions, not just their URLs.

The revised RTCRtpCapabilities would appears as follows:

dictionary RTCRtpCapabilities {
// Indicate which codecs the implementation supports.
    sequence<RTCRtpCodecCapability> codecs;
// Indicate which header extensions the implementation supports.
    sequence<RTCRtpHeaderExtension> headerExtensions;
// Indicate which FEC mechanisms the implementation supports.
    sequence<DOMString>             fecMechanisms;
};

dictionary RTCRtpCodecCapability {
// Provide the name of the codec.
    DOMString                 name = "";
// Tell us what media the codec supports (e.g. "audio", "video" or "" for both) so that getCapabilities(DOMString kind) can return
// a list of codecs supporting the chosen media.
    DOMString                 kind;
    unsigned long?            clockRate = null;
// Provide a preferred Payload Type so as to enable this to be used to configure the PT setting.
    unsigned short            preferredPayloadType;
    unsigned short?           numChannels = 1;
// Need to specify which RTCP feedback mechanisms can be used with which codecs
// This includes transport layer feedback (such as Generic NACK or TMMBR/TMMBN which can be used with any codec)
// as well as Payload-Specific feedback (e.g. PLI, SLI, RPSI) which is utilized and processed by the codec.
    sequence<RTCRtcpFeedback> rtcpFeedback;
// These are parameters that can be used to configure the codec (e.g. fmtp parameters)
    Dictionary                parameters;
// Indicate whether the codec supports Temporal, Spatial and Quality scalability.
    unsigned short            maxTemporalLayers = 0;
    unsigned short            maxSpatialLayers = 0;
    unsigned short            maxQualityLayers = 0;
// Indicate whether a codec supporting SVC can allow each layer to utilize a distinct SSRC or whether all
// Layers need to use the same SSRC.
    boolean?                  multiStreamSupport = false;
};

dictionary RTCRtcpFeedback {
    DOMString type;
    DOMString parameter;
};

dictionary RTCRtpHeaderExtension {
// Indicate what media the header extension can be used with (e.g. "audio", "video" or "" for both)
    DOMString      kind;
// Provide the header extension URI
    DOMString      uri;
// Provide a preferred ID for the header extension so that it can be configured if supported by initiator and responder
    unsigned short preferredId;
// Let us know whether it is preferred for the header extension to be encrypted.
    boolean        preferredEncrypt = false;
};

Based on the above changes, Examples 8 and 9 in the Editor's draft would be replaced with the following:

Example 8

// Assume we already have a way to signal, a transport
// (RTCDtlsTransport), and audio and video tracks. This is an example
// of  how to offer them  and get back an answer with audio and
// video tracks, and begin sending and receiving them.
// The example assumes that RTP and RTCP are multiplexed.
function myInitiate(mySignaller, transport, audioTrack, videoTrack) {
  var audioSender = new RTCRtpSender(audioTrack, transport);
  var videoSender = new RTCRtpSender(videoTrack, transport);
  var audioReceiver = new RTCRtpReceiver(transport);
  var videoReceiver = new RTCRtpReceiver(transport);

// Retrieve the audio and video receiver capabilities
  var recvAudioCaps = RTCRtpReceiver.getCapabilities("audio");
  var recvVideoCaps = RTCRtpReceiver.getCapabilities("video");
// Retrieve the audio and video sender capabilities
  var sendAudioCaps = RTCRtpSender.getCapabilities("audio");
  var sendVideoCaps = RTCRtpSender.getCapabilities("video");

  mySignaller.myOfferTracks({
    // The initiator offers its receiver and sender capabilities.
    "recvAudioCaps": recvAudioCaps,
    "recvVideoCaps": recvVideoCaps,
    "sendAudioCaps": sendAudioCaps,
    "sendVideoCaps": sendVideoCaps
  }, function(answer) {
    // The responder answers with its receiver capabilities

    // Derive the send and receive parameters
    var audioSendParams = myCapsToSendParams(sendAudioCaps, answer.recvAudioCaps);
    var videoSendParams = myCapsToSendParams(sendVideoCaps, answer.recvVideoCaps);
    var audioRecvParams = myCapsToRecvParams(recvAudioCaps, answer.sendAudioCaps);
    var videoRecvParams = myCapsToRecvParams(recvVideoCaps, answer.sendVideoCaps);
    audioSender.send(audioSendParams);
    videoSender.send(videoSendParams);
    audioReceiver.receive(audioRecvParams);
    videoReceiver.receive(videoRecvParams);

    // Now we can render/play
    // audioReceiver.track and videoReceiver.track.
  });
}

Example 9

  // Assume we already have a way to signal, a transport (RTCDtlsTransport)
// and audio and video tracks. This is an example of how to answer an
// offer with audio and video tracks, and begin sending and receiving them.
// The example assumes that RTP and RTCP are multiplexed.
function myAccept(
  mySignaller, remote, transport, audioTrack, videoTrack) {
  var audioSender = new RTCRtpSender(audioTrack, transport);
  var videoSender = new RTCRtpSender(videoTrack, transport);
  var audioReceiver = new RTCRtpReceiver(transport);
  var videoReceiver = new RTCRtpReceiver(transport);

// Retrieve the send and receive capabilities
  var recvAudioCaps = RTCRtpReceiver.getCapabilities("audio");
  var recvVideoCaps = RTCRtpReceiver.getCapabilities("video");
  var sendAudioCaps = RTCRtpSender.getCapabilities("audio");
  var sendVideoCaps = RTCRtpSender.getCapabilities("video");

  mySignaller.myAnswerTracks({
    "recvAudioCaps": recvAudioCaps,
    "recvVideoCaps": recvVideoCaps,
    "sendAudioCaps": sendAudioCaps,
    "sendVideoCaps": sendVideoCaps
  });

    // Derive the send and receive parameters using Javascript functions defined in Section 15.2.
    var audioSendParams = myCapsToSendParams(sendAudioCaps, remote.recvAudioCaps);
    var videoSendParams = myCapsToSendParams(sendVideoCaps, remote.recvVideoCaps);
    var audioRecvParams = myCapsToRecvParams(recvAudioCaps, remote.sendAudioCaps);
    var videoRecvParams = myCapsToRecvParams(recvVideoCaps, remote.sendVideoCaps);
    audioSender.send(audioSendParams);
    videoSender.send(videoSendParams);
    audioReceiver.receive(audioRecvParams);
    videoReceiver.receive(videoRecvParams);

  // Now we can render/play
  // audioReceiver.track and videoReceiver.track.
}