Re: CR exit criteria and features at risk for HTML5

On Wed, 15 Aug 2012, Boris Zbarsky wrote:

> On 8/15/12 2:50 PM, James Graham wrote:
>> 
>> On Wed, 15 Aug 2012, Boris Zbarsky wrote:
>>> Indeed.  Experience shows that the result will most likely be a REC
>>> that can't actually be implemented without breaking web compat, so UAs
>>> will either not follow the REC or errata will be needed on an ongoing
>>> basis.
>> 
>> Is there anyone who believes that won't be the case anyway?
>
> Well, sure, but the question is one of extent of needed errata.
>
>>> Which raises the question of why such a REC is better than a CR that
>>> is updated based on implementation and testing feedback.  I would
>>> really like to understand the reasons why W3C Management thinks it is.
>> 
>> I assume it is because Rec activates parts of the Process related to IPR.
>
> It would be good to have concrete confirmation of this assumption, if this is 
> what's going on here.
>
>> As far as I can tell the most serious danger of the less strict Process
>> is that some people might decide there is less value to them in
>> releasing their tests if they aren't being taken as hard currency in
>> exchange for a Rec.
>
> The most serious danger is that the spec says something ridiculous that no 
> one notices because no one bothers to write tests, and then there is pressure 
> against changing the text "because it's already a REC".  The W3C's record on 
> actually issuing errata to HTML, or for that matter to various other RECs 
> with obvious errors, is less than stellar.

In this case I'm not that worried because the W3C has the WHATWG to keep 
it in line. If the W3C spec can't be trusted on interopeability-affecting 
matters then the WHATWG version will become the de-facto reference for 
implementors. So as long as bugs in the spec are found at all there will 
be strong pressure on W3C to fix their branch or risk becoming irrelevant.

Of course the possibility remains that without "get to Rec." as a carrot 
for writing -- and making public -- tests we will find bugs later than we 
would otherwise have done. But even with specs that are pure greenfield 
development and require a testsuite to get to Rec. people seem to be 
disappointingly reluctant to take the small extra effort to make a public, 
reusable, testsuite compared to a browser-specific testsuite. I would 
happily give up any mention of testsuites in the Process in exchange for 
solving *that* problem.

>> And fixing trivial -- in
>> their effect, but perhaps not in their ease of patching -- bugs that are
>> needed to get through Process hurdles seems strictly less valuable than
>> fixing important bugs that are actually affecting users.
>
> Sure.  The problem is that right now there are large chunks of the HTML5 spec 
> (e.g. the entire page loading and event queue setup) that don't really match 
> any existing implementation, aren't well tested (not least because they're 
> _hard_ to test), and are very likely sources of just the sorts of important 
> bugs you're talking about.
>
>> So in my ideal world, the development of the testsuite continues almost
>> unaffected by the decision here. Certianly I don't foresee it affecting
>> Opera's desire to make our tests public. Possibly it is hopelessly naive
>> of me to think that the same will be true for others?
>
> I think it might be somewhat naive, but the real issue with the loose 
> approach is that it gets us to REC before we can possibly have a reasonable 
> test suite and before we know whether the most risky parts of the spec match 
> reality.

I'm not sure what you consider "the most risky" parts of the spec to be, 
but in my experience the parts least likely to match reality in their 
details are the parts documenting the most fundamental behaviour of the 
web. For example the specification of session history is -- I think -- 
considerably more buggy than the specification of the 2D context or many 
other new features. It's not like we can just drop the session history 
part of the spec, nor are people likely to be happy stalling waiting for 
browsers to rewrite such fundamental code to get to Rec. So there will be 
strong pressure to just drop those tests from the testsuite "for this 
version". So I don't really forsee a testsuite written for the purpose of 
proving interoperability actually helping interoperability as much as a 
testsuite written for the purpose or finding bugs.

Received on Thursday, 16 August 2012 07:05:13 UTC