[HTMLWG] CR Exit Criteria redux

Hello Working Group,

A little while back, we discussed possible CR exit criteria for our specs. The WG seemed largely in favor of the Public Permissive exit criteria proposed here: <http://lists.w3.org/Archives/Public/public-html/2012Aug/0294.html>. We recently reviewed these with W3C Management, so see if this style of CR exit criteria was likely to fly (since they would be reviewed by the Director when we propose entry to CR). We got a few points of feedback that I thought were interesting and worth sharing with the group:

(1) If we post a Call for Consensus to adopt these, we should be clear that these are the default, but current and future extension specifications may use different criteria (for example, if they have a different set of relevant conformance classes).

(2) These criteria are unclear on whether the combination of a user agent with another piece of software, such as an assistive technology, or a browser extension, would qualify for purposes of meeting the CR exit criteria. My sense is that as long as you have two fully independent combo stacks. In other words, two extensions to the same browser, or extensions for multiple browsers sharing code, would not constitute fully independent implementations. In addition, the add-on pieces of software meet the "non-experimental" condition (i.e. they are legitimately intended and not created solely to meet the CR exit criteria), then that would be acceptable. It's good to have expectations about this clear up front.

(3) We should define what "at a judgment level" means better, and how it relates to testing.

I've attempted to apply (3) to the version below. For (1) and (2) I'd like to see some discussion, and comments on whether there is anything to add to the CR exit criteria themselves.

---------------

=== Model CR Exit Criteria (Public Permissive version 2)

For this specification to be advanced to Proposed Recommendation, there must be at least two independent, interoperable implementations of each feature. Each feature may be implemented by a different set of products, there is no requirement that all features be implemented by a single product. For the purposes of this criterion, we define the following terms:

== independent

Each implementation must be developed by a different party and cannot share, reuse, or derive from code used by another qualifying implementation. Sections of code that have no bearing on the implementation of this specification are exempt from this requirement.

== interoperable

Qualitatively interoperable at at a judgment level, not necessarily for every spec assertion. A test suite may be used as guidance for the qualitative decision.

== implementation

A user agent which: (1) implements the "Web browsers and other interactive user agents" conformance class of the specification. (2) is available to the general public. The implementation may be a shipping product or other publicly available version (i.e., beta version, preview release, or “nightly build”). Non-shipping product releases must have implemented the feature(s) for a period of at least one month in order to demonstrate stability. (3) is not experimental (i.e., a version specifically designed to pass the test suite and is not intended for normal usage going forward).

== judgment level

For features that are well known to be widely implemented and deployed, the Working Group will assume effective real-world interoperability without testing. For features that are known not to be implemented at all, or unlikely to have multiple implementations during the CR period, the Working Group will not invest significant effort in testing. For features where the interoperability level is uncertain, the Working Group will create a sufficient test suite to make a judgment call; 100% pass rate will not necessarily be required. In any case where the judgment is debatable, it will be a Working Group decision whether sufficient interoperability has been achieved.

The specification will remain Candidate Recommendation for at least six months.

The following features are at risk: <fill in all features that do not already have two qualitatively interoperable implementations on entry to CR>.

Received on Saturday, 15 September 2012 01:12:43 UTC