RE: Testing for WebRTC

I am in favor of what Dom proposed:

"
... I think the WebRTC Working Group should include in its CR exit criteria actual interop across actual browsers, which we should prove with testing across different browsers ...
"

According to what is indicated in the CR Exit Criteria [1]:

An implementation means "A user agent which: (1) implements the "Web browsers and other interactive user agents" conformance class of the specification. (2) is available to the general public. The implementation may be a shipping product or other publicly available version (i.e., beta version, preview release, or “nightly build”). Non-shipping product releases must have implemented the feature(s) for a period of at least one month in order to demonstrate stability, or be endorsed by their responsible organization as sufficiently stable. (3) is not experimental (i.e., a version specifically designed to pass the test suite and is not intended for normal usage going forward). (4) is suitable for a person to use as his/her primary means of accessing the Web."

So in my view, "a partial, instrumented implementation that could be used for testing" is not qualified for an "Implementation" in this CR Exit Criteria context. And "a reference implementation" is not qualified either unless this reference implementation can meet all of the above 4 criteria of an "Implementation".

Thanks
Bin

[1] http://dev.w3.org/html5/decision-policy/public-permissive-exit-criteria.html


-----Original Message-----
From: Dominique Hazael-Massieux [mailto:dom@w3.org] 
Sent: Wednesday, October 23, 2013 5:41 AM
To: Tobie Langel
Cc: Robin Berjon; public-test-infra@w3.org
Subject: Re: Testing for WebRTC

Le mercredi 23 octobre 2013 à 14:38 +0200, Tobie Langel a écrit :
> My understanding of what Robin describes is a partial, instrumented
> implementation that could be used for testing.
> 
> If so, it could qualify as an interoperable implementation, no?

I'm not I understand this; let me try to explain what I mean: when a new
phone protocol goes out, implementors not only tests their
implementations against a reference implementation, they also test that
a real phone A can indeed call a real phone B and vice versa.

I'm saying the same type of interop testing needs to be done for WebRTC.

Dom

Received on Wednesday, 23 October 2013 14:23:38 UTC