Re: snapshots vs living standards

On Fri, 02 Mar 2012 19:27:45 +0100, Philippe Le Hegaret <plh@w3.org> wrote:

> Hi All,
>
> I noticed the following post from Ian Hickson and thought it is relevant
> for this list. I'm copying the content here to facilitate the
> discussion...

Thanks. I agree with Robin that there are some important points and some
nonsense in here. So I've attempted to seperate them inline. I'd like to
tease out more clearly deineated issues, but that will take some time I
don't have right now... so this is mostly for Marcos' benefit I guess :)

> In a recent private discussion spawned from one on a W3C mailing list, I
> was defending the "living standard" model we use at the WHATWG for
> developing the HTML standard (where we just have a standard that we
> update more or less every day to make it better and better) as opposed
> to the "snapshot" model the W3C traditionally uses where one has a
> "draft" that nobody is supposed to implement, and when it's "ready",
> that draft is carved in stone and placed on a pedestal.

Actually, I think what happens is that the document is published in a
stable form. There is a process for publishing errata, or a revised
edition, although that seems not to be used in practice as much as it
should be (and I think that is a problem, but not one that the 'living
standard' model addresses very well).

There is also a process for continuing work on a "new version", and that
seems to work - we have XHR (which simply dropped the XHR1 work), HTML5
(after 4 and 3.2 and the IETF's 2.0), CSS 4 modules appearing (with CSS 3
modules developing, CSS 2.1, 2 and 1 having been published and apparently
put to bed), XML having been revised a few times, etc etc.

I think a more sensible charaterisation is between a stabilising
publication process (I'll continue to call this the 'snapshot model' here
because I doubt that trying to change terms is useful), and one where
changes can just be thrown in at any time.

> The usual argument is something like "engineering depends on static
> definitions, because otherwise communication becomes unreliable".

It seems that we have been discussing in different circles then. The
arguments I usually come across turn around the ability to easily identify
the same definition that is stable enough to use as a basis for a
Statement of Work, or to complete a thorough review of a very complex
specification, or to clearly identify a bug and describe a change.

I believe the idea of "static definition" is misguided (and some of the
arguments Ian produces below illustrate why), and I don't believe it is
particularly relevant to W3C's work which involves maintaining
specifications for technology that is in development.

> I wrote a lengthy
> reply. I include it below, in case anyone is interested.

> With a living standard, one cannot change things arbitrarily.

This is nonsense. As implied hereafter by Ian, one *should* not and I
doubt there is disagreement with that.

> The only possible changes are new features, changes to features that
> aren't implemented yet

These changes can be made in any model. And are important, obviously
(hence the series of versions for so many W3C specs).

> or that have only had experimental implementations,

The definition of "experimental" is a practical question more important
than implied here. In practice, features from "experimental"
implementations are regarded as too important to the existing world to
change, and "non-experimental" implementations can be called experimental
(or irrelevant) for reasons that owe as much to politics as technical
merit when justifying a change.

(Examples of the former might include some -webkit- prefixed CSS
properties, examples of the latter might include longdesc or widgets).

Having already concluded that changes *can* be made, but should not be
made arbitrarily, I think we can defer this point to a separate discussion
based on the restrictions of topic Ian makes below.

> and changes to bring the specification even more in line with what is
> needed for reliable communication, as the argument above puts it (or
> "interoperability", as I would put it), i.e. fixing bugs in the spec.

Right. What the existing Errata / Revised specification process is meant
to deal with in W3C.

> With a snapshot-based system, like the W3C TR/ page process, adding new
> features is done by creating a new version,

This is nonsense. A new feature *can* be introduced during the development
of a specification. The W3C Patent Policy requires that all features are
present in a last call - so anythign added after that to an existing
version means that version has to go through last call again (There *is*
an issue with how this happens currently at W3C, but it isn't clear that
"living standards" are any good as a solution). Nothing in W3C Process or
practice prevents new versions being developed, and in many cases this is
done simultaneously, so leaving it out of the discussion has an effect of
"gerrymandering the topic" (potentially justifying a conclusion without
properly examining the issues it purports to resolve).

> so we'll ignore that, and experimental stuff is taken out before taking
> the snapshot, so we'll ignore that too.

This is nonsense too, and leaving it out of the discussion has the same
effect as above. Marking features "at-risk" in CR documents is precisely
for the case where something is considered experimental, yet has enough
support to get into a CR with the patent support that implies from the WG.

> That leaves the fixing bugs.

Not really...

But let's follow that part of the discussion to see what we can learn
 from it. As argued above, this in practice corresponds only to the Errata
and Edited Recommendation parts of W3C process, but that's OK, they are
legitimate topics for discussion.

> Now either one doesn't fix the bugs, or one does fix the bugs.

Let's assume for now that one doesn't just mess around with them and make
slightly different bugs, to keep the discussion simple.

So provisionally, yes.

> If one fixes the bugs, then that means the spec is no more "static"
> than a living standard.

More nonsense. A "late-stage" W3C spec is more stable by virtue of being
changed only under defined conditions which are rather more predictable
than "the editor woke up today with a new brainwave"

It seems that the working assumption here is that W3C's "snapshot model"
requires specs not to be changed. (Unsurprisingly given that W3C has been
evolving specs for most of a couple of decades) that contradicts what
W3C's Process says, and is at odds with the real-world practice.

W3C specifications, especially in post-experimentation stages, have
specific restrictions on the process of changing things in order to make
them more stable than "living standards" - which is in turn used for
things like getting companies to agree to license IPR they have paid to
develop and identify for free. This does indeed have a cost.

I am not alone in considering it to also have considerable benefit. (I
also believe I am not wrong about the benefits overall, but that could
be a more valuable discussion).

> And if one doesn't fix the bugs, then communication becomes unreliable.

This is not necessarily true - although generally, fixing the bugs does
improve communication, and is a good idea.

Conversely, relying on everyone who is implementing HTML also following
the discussions everywhere is nonsense, and assuming that they follow
daily change reports is at best a dubious prospect. I work for an
implementor who treats HTML 5 as absolutely central to their business,
where every employee speaks good english, where we have a high degree of
active involvement in standards work, and people specifically assigned to
watch important specifications. And even in such an ideal case, there are
situations where communication is unreliable, and the instability of a
daily-changing draft generally makes it more so. Many people implementing
HTML 5 don't have all the luxuries we do, and I

> So IMHO, it's the snapshot-based system that's the one that leads to
> unreliable communications.

IMHO, if this opinion is based on the above information, it's a rash
conclusion.

> Engineering doesn't depend on static definitions, it depends on accurate
> definitions.

Let's provisionally agree on this, as it probably means roughly the same
to all of us, and isn't a priori wrong.

> With a platform as complicated as the Web, we only get accurate
> definitions by fixing bugs when they are found,

But this is. We get accurate definitions by having clear communication of
those definitions. For some bugs, just fixing them as soon as you can is
the best thing to do, with the greatest benefit and least adverse impact.
For other bugs (e.g. that the CSS Box model wasn't written clearly enough
for Microsoft's engineers to understand it, or whatever happened there)
the likely impact of Just Fixing It™ is that the world communicates about
something (the "proper" box model) that doesn't match reality. And we have
therefore broken the requirement that we have accurate definitions - by
using the living standard model.

> which inherently means that the definitions are not static.

Which is irrelevant. The question is not whether definitions are or should
be static.

> Note that we know this system works. HTML has been developed in a
> "living standard" model (called "working draft" and "editor's draft" by
> the W3C) for seven and a half years now. Interoperability on the Web has
> in that time become dramatically better than it ever was under the old
> "snapshot" system.

Clearly, HTML being developed in an incremental model with no "end-game"
is a more probable cause for the improvement in interoperability than the
fact that I have been working for Opera during that period. But that's not
really a convincing argument. W3C has been developing HTML under its
"snapshot-based" model for about 5 years - and I believe that the vast
bulk of the improvement in interoperability came in that time. Large
players like Microsoft have explicitly avoided the "living standard"
version, and that appears not to have slowed the improvement.

> Also, note that the snapshot system without applying fixes, which is the
> system that the W3C actually practices on the TR/ page (very few specs
> get errata applied, even fewer get substantial in-place corrections in
> any sort of timely manner), has demonstrably resulted in incorrect
> specs. HTML4 is the canonical example of this, where in practice that
> spec is woefully inaccurate and just poorly written, yet nothing was
> ever done about it, and the result was that if you wanted to write a
> browser or other implementation that "communicated reliably" with other
> HTML UAs, you had to explicitly violate the spec in numerous ways that
> were shared via the grapevine (the default value of the "media"
> attribute is the example I usually give of this).

It is reasonable to argue that HTML4 was not a well-written spec, and I
think generally accepted that it did not match reality very well. Indeed,
W3C changed its process significantly shortly afterward, to ensure that
"finished" specs would match reality more closely.

> And also, note that having a static specification doesn't ensure that
> nothing will ever change.

Yes. I believe this point is not disputed, and that it is irrelevant.

> HTML is a stark example of this, where from HTML4 to the contemporary
> specification many things have changed radically. But you might say
> that's my fault,

No, I would suggest that Ian is incidental to that. HTML changed
enormously between HTML 3.2 and HTML 4 as well, under the work-in-private,
waterfall-style, snapshot-based publishing process then in place. HTML 3.2
was also a significant change to HTML. And to the extent that HTML 4 was a
failure in not being exactly implemented, HTML 3.2 was in very many ways an
enormous success. It has masses of implementations, but admittedly HTML was
simpler then.

> so look instead to XML:
> the specification has changed such that implementations of the original
> standard are no longer conforming to the current standard. In fact, XML
> is arguably developed in a kind of "living standard" approach now,
> despite officially using the snapshot model.

The snapshot model doesn't preclude change. In the form W3C practices it,
certain constraints are placed on the rate of change, and there are
certain marker points distinguishing particular drafts. Whereas in the
way WHAT-WG practices the living standard model, the differences between
different drafts can be of an arbitrary nature, and the only way to
understand them is to read the changelogs, and in many cases also the
discussion that explains each of the changes made. This strikes me as
a weakness of the living standard model.

The argument here suggests to me that the W3C model actually *works*,
getting bugs fixed and having clearly defined finished specifications that
allow interoperable implementations to be built against them, and that
improve over time providing a basis for improving the interoperability of
those implementations.

As a reference, would anyone care to expound on the relative
interoperability of XML tools and the Web over the last decade or so? it
isn't an area I know really well.

> Things change, especially in software. And that's ok!

Sure.

> But it's incompatible with the snapshot model. Or at least,
> interoperability is made harder with the snapshot model.

There are drawbacks to the snapshot model other than those explained here.
There are benefits too, and I don't think they have been explained either.

I certainly don't find any significant argument here to suggest W3C should
simply abandon its approach and move the work to WHAT-WG, beyond "they
messed up by not doing HTML5 earlier". I think everyone would agree that
was a historic mistake. I find it hard to understand the leap from there
to "therefore they are doing everything wrong" and I don't see a logically
constructed argument to that effect.

> </soapbox>
>
> Before commenting on this, please check to see if your argument is
> already refuted on the WHATWG FAQ: http://wiki.whatwg.org/wiki/FAQ

(I did).

> ]]
> https://plus.google.com/107429617152575897589/posts/NZBJe6Jjt1f
>
> Philippe
>
>


-- 
Charles 'chaals' McCathieNevile  Opera Software, Standards Group
       je parle français -- hablo español -- jeg kan litt norsk
http://my.opera.com/chaals       Try Opera: http://www.opera.com

Received on Monday, 5 March 2012 10:49:53 UTC