Costs and Benefits of Early Implementations (was Re: The harm that can come if the W3C supports publication of competing specs)

On Jan 17, 2010, at 5:58 PM, Boris Zbarsky wrote:

> On 1/17/10 5:36 PM, James Graham wrote:
>> So it seems like you are saying that, in this case, a mostly-unrelated
>> shift in the technology landscape (browsers wanting to switch to multi
>> thread/process) designs caused problems with an existing feature.
> 
> Sort of.  This was the case for Gecko.  In Chrome's case, the multi-process was in fact done after localStorage, without worrying overmuch about the issues as far as I can tell.  I have no idea what the ordering was for IE, since both shipped in IE8 and their development process is somewhat more opaque than Webkit's or Gecko's...

I think the timeline was in this order:

1) Initial draft of LocalStorage goes in the spec and sees some refinement.
2) Google Chrome begins secret development and implements multiprocess. The existence of Chrome and its multiprocess nature are a closely guarded secret.
3) WebKit implements LocalStorage in the public tree, but Chrome doesn't merge or adapt the changes before initial release.
4) Chrome engineers adapt WebKit's LocalStorage implementation to work with their multiprocess model and discover issues.
5) Chrome engineers report the problem.

So with regards to Chrome, matters are as James describes. In theory, if Google had shared more of their plans sooner or integrated their multiprocess work directly into WebKit, we might have known about the problems sooner, and potentially might have dealt with it before IE shipped. But it would have been an accident of fate.

>> I strongly suspect that only an actual shipping implementation was good
>> enough. I imagine people are not that interested in authoring production
>> sites against "experimental" features that only appear in
>> technology-preview builds (or even features that are prefed off by
>> default in shipping builds).
> 
> Maybe.  We (Gecko) get a fair number of bug reports against such features when we have them.  Clearly not a substitute for widespread author deployment, but much better than shipping first and getting feedback later.  Hard to say whether that would have caught the autobuffer issue, of course; you may be right about that particular case.

New WebKit features tend to appear in WebKit nightlies well before they ship in any production browser and we do get a fair amount of feedback. But that's dwarfed by the level of detailed feedback if we ship it in a release, or if some content author tries to use it in a production site.

We try to avoid shipping or encouraging use of things that are overly half-baked, and at times we disable features for releases that are checked in on trunk and not fully baked. But we can't keep features restricted to experimental builds indefinitely.

> 
>>> For the full draft, we may be sort of stuck with issue #2 because the
>>> situation is complex to describe (though saying "this is all unstable
>>> unless explicitly indicated otherwise" might help). That's not a
>>> restriction we have with standalone drafts.
>> 
>> The draft already says exactly that:
>> 
>> "Implementors should be aware that this specification is not stable.
>> Implementors who are not taking part in the discussions are likely to
>> find the specification changing out from under them in incompatible
>> ways.
> 
> That doesn't tell web authors "This is half-baked; don't push for implementations of it."

Individual sections of the spec are marked with a maturity level in both W3C and WHATWG copies of the spec. I don't think it's the place of the spec to go further and tell content authors what they should or should not be asking of browser developers.

>> I don't think that slicing and
>> dicing the spec into a multitude of different specs fundamentally
>> changes the market dynamics; implementers implement the things that they
>> think are best for the platform (i.e. the things that authors are asking
>> them for).
> 
> Maybe.  Implementers implement what they feel like; as far as I can tell, some of it is for the reason you describe and some is to be able to check off checkboxes in a "we support hot new technology X" spreadsheet that they can then hype.  Never mind if their implementation is broken, or even useless.  The important part is that its "implemented".
> 
> We don't want to encourage that dynamic, and right now we are, with our de-facto policies of not changing stuff once someone ships it and our unclear message to web authors about what things they should push for implementation of and which they shouldn't.  The net result is there is at least short-term competitive advantage to shipping crappy implementations of as much of "HTML5" as possible as fast as possible, whatever the effects on "the platform".  Now on the other hand all the implementors involved happen to be decent human beings who don't want to poison features, etc.... and that's what keeps the whole thing from being a total fiasco.
> 
> And this issue is not necessarily restricted to HTML5 (some CSS features have similar issues).  Basically, any time implementation happens to check off a checkbox, the result seems to be brokenness.

I agree that these are real problems, but I'm not hearing a better alternative model. Waiting longer to implement features (until CR, say) likely just means that many problems stay undiscovered for longer. Keeping new features restricted to experimental builds greatly reduces the level of testing and review by content authors, and also becomes unsustainable as the number of not-yet-finalized features grows. Breaking deployed content freely is something we should not be eager to do. My feeling, as I've stated before, is that deployed content and implementations should be a factor we weigh in deciding whether a change is worth it, but not a determining factor by itself. We need to balance the scale of deployed content against the benefit  of change.


> 
>> These are often the things that other implementers have
>> already implemented and so are needed for compatibility.
> 
> That's a separate story entirely; the parts of HTML5 that are "needed for web compat" are not the parts I'm worried about here.
> 
>> All these pressures are more-or-less independent of anything in the spec
>> itself and of anything that the W3C, the WHATWG, or anyone else in specs
>> land, does.
> 
> I honestly don't think that's the case.
> 
>> I guess we could start a massive negative publicity campaign
>> about our own spec and talk about how it won't be ready until 2022 or
>> something but, quite apart from the fact that that seems silly, the
>> "HTML5 won't be ready for many years" thing has already happened and
>> hasn't stopped people using the features that are shipping already. I
>> guess implementers could band together and pledge not to implement
>> certain things, but it hardly seems likely to happen and doesn't even
>> seem particularly likely to improve the overall quality of the spec when
>> it finally is implemented.
> 
> I think we could effectively focus on getting those parts of the spec more likely to be implemented in the near future into a stable state by giving Ian feedback that he should focus on them and that we could focus a lot more on providing implementation feedback while implementing, and not sometime shipping.  I don't think we've done a great job of this, as a group.  Gecko's implementors of the offline cache stuff didn't ask nearly as many question as I think they should have.  I don't recall a slew of questions about web forms coming through, and the stuff there is complex enough that I find it a little hard to believe that there were no questions to be asked.  That sort of thing.
> 
> Of course I could just be completely off my rocker here.  Wouldn't be the first time!

For what it's worth, almost every significant new HTML5 (or HTML5 spin-off) feature that's gotten implemented in WebKit has resulted in a significant level of questions and feedback, either here or via the WHATWG. In some cases we could have had more. But we definitely set the expectation that whenever a WebKit contributor adds an early implementation of a new standards feature, part of their job is to give feedback to the relevant standards body. Similarly, when we invent new features ourselves, part of the process of getting it landed involves figuring out our strategy for getting it into standards and for not overly poisoning whatever standard results.

I would encourage all implementors to think in these terms, but I recognize that there's a limit to how much we can dictate their actions.

Regards,
Maciej

Received on Monday, 18 January 2010 02:21:52 UTC