RE: Issue 236 (API for CSRCs and audioLevels)

Here is an updated proposal


partial interface RTCRtpReceiver : RTCStatsProvider<http://internaut.com:8080/~baboba/ortc/ortc-9-18-2015.html#idl-def-RTCStatsProvider> {

    sequence<RTCRtpContributingSource<http://internaut.com:8080/~baboba/ortc/ortc-9-18-2015.html#idl-def-RTCRtpContributingSource>> getContributingSources ();

};:

getContributingSources
Returns an RTCRtpContributingSource object for each unique CSRC or SSRC received by this RTCRtpReceiver since the last time this method was called.
No parameters.
Return type: sequence<RTCRtpContributingSource>

dictionary RTCRtpContributingSource

The RTCRtpContributingSource object contains information about a contributing source. Each time an RTP packet is received, the RTCRtpContributingSource objects are updated. If the RTP packet contains CSRCs, then the RTCRtpContributingSource objects corresponding to those CSRCs are updated, and the level values for those CSRCs are updated based on the mixer-client header extension [RFC6565] if present. If the RTP packet contains no CSRCs, then the RTCRtpContributingSource object corresponding to the SSRC is updated, and the level value for the SSRC is updated based on the client-mixer header extension [RFC6464] if present. If the RTP packet does not contain a client-to-mixer header extension, then the browser will compute the level value as described in [RFC6464] and will provide that.

interface RTCRtpContributingSource {

    readonly    attribute DOMHiResTimeStamp timestamp<http://internaut.com:8080/~baboba/ortc/ortc-9-18-2015.html#widl-RTCRtpContributingSource-timestamp>;

    readonly    attribute unsigned long     source<http://internaut.com:8080/~baboba/ortc/ortc-9-18-2015.html#widl-RTCRtpContributingSource-source>;

    readonly    attribute byte?             audioLevel<http://internaut.com:8080/~baboba/ortc/ortc-9-18-2015.html#widl-RTCRtpContributingSource-audioLevel>;

};

6.4.1 Attributes
audioLevel of type byte, readonly , nullable

The audio level contained in the last RTP packet received from this source. If the source was set from an SSRC, this will be the level value in [RFC6464]. If the source was set from a CSRC, this will be the level value in [RFC6465]. Both [RFC6464] and [RFC6465] define the level as an integral value from 0 to -127 representing the audio level in decibels relative to the loudest signal that they system could possibly encode.
source of type unsigned long, readonly

The CSRC or SSRC value of the contributing source.
timestamp of type DOMHiResTimeStamp, readonly

Time of reception of the most recent RTP packet containing the contributing source. The time (obtained on the user agent) is relative to the UNIX epoch (Jan 1, 1970, UTC).



From: Bernard Aboba [mailto:Bernard.Aboba@microsoft.com]
Sent: Friday, September 18, 2015 4:45 PM
To: Robin Raymond <robin@hookflash.com>; public-ortc@w3.org
Subject: RE: Issue 236 (API for CSRCs and audioLevels)

Robin said:

“I would actually recommend the API return all activity SINCE the last call to the API. Thus each time the API is called, the old list is purged and a new list is built thus allowing the application to keep all information about the contributing sources without loss but still not maintain huge volumes of data needlessly over time”

[BA] I believe that was the original intent of this API, which is why the timestamp made sense.  That formulation discourages rapid polling – although the UI might not be as responsive, you won’t be missing information if you call the API every 5 seconds versus every second.   Whereas if entries older than a second (or 10 seconds, for that matter) age out, I agree that the timestamp loses its value.

I would also question the value of returning both SSRC entries and CSRC entries within the sequence of RTCContributingSources.

In the mixer use case, we need CSRCs and audio Level values for participants in the conference.  The level of the mixed audio stream is not relevant to whether individual participants are contributing sources or to the value of their contributing audio levels.

In the P2P case, we need the SSRC and the level is provided by the client-mixer extension (RFC 6464, mandatory-to-implement in RTP-Usage).  In this use case there is no mixer-client extension (RFC 6465) or CSRCs (since there is no centralized mixer).

So if we are focusing on these two use cases, we will either have an SSRC dictionary entry, or you have a sequence of dictionaries for the CSRCs.



From: Robin Raymond [mailto:robin@hookflash.com]
Sent: Friday, September 18, 2015 4:07 PM
To: public-ortc@w3.org<mailto:public-ortc@w3.org>; Bernard Aboba <Bernard.Aboba@microsoft.com<mailto:Bernard.Aboba@microsoft.com>>
Subject: Re: Issue 236 (API for CSRCs and audioLevels)


“The browser must keep information from RTP packets received in the previous second. “

In this context, I’m unclear the value of the timestamp into the contributing sources. The timestamp can only have a resolution sub-second and that will tell you relative to the last second when they last contributed. Not sure knowing that someone was 450 ms vs 210 ms difference is all that important. Only keeping track within 1 second limits the usefulness of the levels to having flickering LED activity lights and the relative timestamp is not valuable.

If the browser kept a much longer history then we could tell whom is currently active vs whose been inactive for a long time. That might allow us to not only do flickering LEDs but to eliminate people whom have been inactive for a long period of time.

I would actually recommend the API return all activity SINCE the last call to the API. Thus each time the API is called, the old list is purged and a new list is built thus allowing the application to keep all information about the contributing sources without loss but still not maintain huge volumes of data needlessly over time.

-Robin



On September 11, 2015 at 3:09:59 PM, Bernard Aboba (bernard.aboba@microsoft.com<mailto:bernard.aboba@microsoft.com>) wrote:

[BA] Some questions:

1.    When an RTP packet is received with a Mixer-Client extension and CSRCs is an RTCRtpContributing source dictionary created for the SSRC as well as the CSRCs?   If so, is an audioLevel attribute computed with that dictionary?   Or is the audioLevel only computed for the SSRC when there are no CSRCs?

2.    If an RTP packet is received with no Mixer-Client extension, but with CSRCs, would the RTCRtpContributingSource dictionaries for the CSRCs have the audioLevel attribute unset since that cannot be determined?

3.    How is the timestamp value obtained?  Is this the timestamp value from the RTP packet?  Or is it an NTP timestamp?



=====================================================================================

partial interface RTCRtpReceiver : RTCStatsProvider {

    sequence<RTCRtpContributingSource> getContributingSources ();

};


getContributingSources
Returns an RTCRtpContributingSource for each unique CSRC or SSRC received by this RTCRtpReceiver in the last second.
No parameters.
Return type: sequence<RTCRtpContributingSource>


6.4 dictionary RTCRtpContributingSource

The RTCRtpContributingSource object contains information about a contributing source, such as the most recent time a packet was received from the source. The browser must keep information from RTP packets received in the previous second. When an RTP packet is received, the entries are updated for the SSRC of the RTP packet and for any CSRC contained within it.



interface RTCRtpContributingSource {

    readonly    attribute DOMHiResTimeStamp timestamp;

    readonly    attribute unsigned long     source;

    readonly    attribute byte?             audioLevel;

};

6.4.1 Attributes
audioLevel of type byte, readonly , nullable
The audio level contained in the last RTP packet received from this source. If the source was set from an SSRC, this will be the level value in [RFC6464]. If the source was set from a CSRC, this will be the level value in [RFC6465]. If the RTP packet does not contain a client-to-mixer header extension, then the browser will compute the level value as described in [RFC6464] and will provide that. Both [RFC6464] and [RFC6465] define the level as a integral value from 0 to -127 representing the audio level in decibels relative to the loudest signal that they system could possibly encode.
source of type unsigned long, readonly
The CSRC or SSRC value of the contributing source.
timestamp of type DOMHiResTimeStamp, readonly
Time of reception of the most recent RTP packet containing the contributing source.