W3C home > Mailing lists > Public > public-webrtc@w3.org > September 2012

Re: Generalizing the stats hierarchy

From: Harald Alvestrand <harald@alvestrand.no>
Date: Thu, 27 Sep 2012 16:49:44 +0200
Message-ID: <50646788.5020203@alvestrand.no>
To: public-webrtc@w3.org
(back to being serious on this thread, and top-posting because I want to 
talk about general principles rather than details of a proposal..)

My experience with stats systems (mostly SNMP, but also mrtg, nagios and 
a couple of Google-internal systems) is that they generally descend to 
work on basic items, and offer powerful graphing, summarization and 
alarming features that work from these primitive values - but rarely, if 
ever, have I found an interface where it's comfortable to work with 
structured values directly.

I *like* structured objects. They are extremely useful in many ways, 
especially when thinking about things, but they are hard to handle. In 
particular, in the WebIDL type system, having functions that return 
"either an object or an array or a dictionary" is tricky.
In SNMP, they finessed the issue by marking every object that is not a 
primitive type with the magical incantation "max-access not-accessible".

Martin's mention of "JSON pointer" (I assume that this is 
draft-ietf-appsawg-json-pointer-03.txt) makes me think that we might be 
able to have our cake and eat it... if we define just 2 operations:

- getValue("identifier") -> primitive object
- getNames("identifier") -> list of names (sequence<DOMString>) that are 
valid for the next level down

we can represent any level of complexity, and allow navigation through 
it, without having to deal with compound values.
(Of course, we can also allow getValue to return compound values - if 
anyone cares enough to get Webkit and friends to understand that.....)

The objects I see are:

- SSRCs (one or more per MediaStreamTrack - that is, an N:1 mapping)
- Transports (1:N mapping to SSRCs)
- Components (RTP / RTCP ... where does DTLS fit?)
- Candidate pairs (N:1 mapping to transports)
- Candidates (N:N mapping to candidate pairs??)

Offhand, I don't see a need to have complex stats on any of these, but 
others might....


On 09/24/2012 06:05 PM, Martin Thomson wrote:
> This makes sense to me.  Far more so than the flat structure.  It
> seems clear that the cost of managing structure warrants the
> (marginal) extra complexity that this results in.
>
> Did you consider JSON Pointer as a way to identify nodes in a tree?
> Alternatively, you could expose the entire tree in the report without
> any need for getValue():
>
> Your example:
>    report.getValue("ICE.0")
> JSON pointer:
>    report.getValue("/ICE/0")
> DIrect:
>    report.value.ICE[0]
>
> These are, after all, just dictionaries and sequences of things.  A
> direct approach allows for inspection with things like
> hasOwnProperty(), the "in" operator, forEach(), and so forth.  Much
> easier to program to.
>
> On 24 September 2012 08:13, Eric Rescorla <ekr@rtfm.com> wrote:
>> Harald,
>>
>> In draft-alvestrand-rtcweb-stats-registry-00.txt, you observe that
>> there are times when a single named statistics value actually
>> corresponds to a number of elements and you would like to be able to
>> address them individually. You suggest handling this case with the
>> convention of appending a ".X" to the stat in question, but
>> I think this actually points to the need towards genuinely
>> hierarchical stats.
>>
>> Consider the case where you want to examine every aspect of ICE,
>> which I think there is general consensus we need. At this point
>> we have the following containment hierarchy:
>>
>>    - Media Stream  [W3C name: track]
>>    - Component     [RTP or RTCP]
>>    - Local candidate
>>      - State
>>      - Check history
>>      - Estimated RTT
>>
>> This seems pretty deep to represent cleanly in the existing hierarchy
>> but would fit well into a more generic structure.
>>
>> Here's a strawman to give you an idea of what I have in mind:
>>
>> - Instead of being just opaque strings, stats identifiers
>>    should be dot-separated strings, with dots separating
>>    levels in the hierarchy.
>>
>> - When registered, each stats identifier must be one of:
>>
>>    * value -- the value is in the stat itself
>>    * array -- the stat contains a list of values in an array
>>      (i.e., [])
>>    * dictionary -- the stat contains a list of values in a
>>      dictionary (i.e., {})
>>
>> - You can call getValue() at any level in the hierarchy
>>    and what you get depends on the identifier type. You
>>    can subaddress arrays and dictionaries by including
>>    the index/key in the identifier (as shown below).
>>
>>
>> Reworking your ICE example in this fashion would give us something like this:
>>
>> { local: { timestamp: 12345, stats: {
>>           SentPackets: 47,
>>           SentOctets: 4444,
>>           ReceivedPackets: 33,
>>           ReceivedOctets: 2346,
>>           ICE: [
>>             {
>>               State: Succeeded
>>               Used: True,
>>               LocalIpAddr: '129.241.1.99',
>>               RemoteIpAddr:'234.978.4.3'
>>             },
>>             {
>>               LocalIPAddr: '10.0.0.1',
>>               RemoteIPAddr: '10.0.1.24',
>>               State: Succeeded
>>               Used: False
>>             }
>>           ]
>> }}}
>>
>> ISTM that this places things that naturally go together together,
>> and also makes it easier to build processing engines without a lot
>> of string manipulation.
>>
>>
>> If I am reading the current API correctly, the only way to actually
>> get at a statistics value is to do .getValue() on an RTCStatsReport.
>> In this case, the code would then be something like this:
>>
>>     report.getValue('SentPackets') --> 47
>>     report.getValue('ICE') -->
>>           ICE: [
>>             {
>>               State: Succeeded
>>               Used: True,
>>               LocalIpAddr: '129.241.1.99',
>>               RemoteIpAddr:'234.978.4.3'
>>             },
>>             {
>>               LocalIPAddr: '10.0.0.1',
>>               RemoteIPAddr: '10.0.1.24',
>>               State: Succeeded
>>               Used: False
>>             }
>>           ]
>>
>>
>>     report.getValue('ICE.0') -->
>>             {
>>               State: Succeeded
>>               Used: True,
>>               LocalIpAddr: '129.241.1.99',
>>               RemoteIpAddr:'234.978.4.3'
>>             }
>>
>>     report.getValue('ICE.0.State') --> 'Succeeded'
>>
>> Thoughts?
>> -Ekr
>>
Received on Thursday, 27 September 2012 14:50:19 GMT

This archive was generated by hypermail 2.2.0+W3C-0.50 : Thursday, 27 September 2012 14:50:20 GMT