RE: New resource: Normative References to W3C Standards

I want to reply to an earlier message in the thread, to try to keep the topic focused on the one use case normative references.
The TAG took up the issue in 2009
http://lists.w3.org/Archives/Public/www-tag/2009Oct/0075.html

and later I took it as an action to develop an update
https://www.w3.org/2001/tag/group/track/actions/350

which was  resolved it by pointing to a previous W3C recommendation
http://www.w3.org/TR/qaframe-spec/#ref-define-practice


based on multiple discussions. Perhaps updating  http://www.w3.org/wiki/NormativeReferences would be useful.

I think this is an important question to resolve in the W3C.

I think you are using the term "stable" in a very different way than I do, and I claim than "most" do.  I'm willing to distinguish:

 - static specifications (document doesn't change)
 -  stable specifications (only non-technical editorial changes)
 - "latest version" specifications,

and would like to assert only that  there are different user communities who might have different needs for these,
and asks that we focus on  the main question whether there are hard, soft, or no rules for normative references among the categories.

I'm not sure everyone agrees with this formulation of the problem, so please speak up whether or not you agree.
If you don't, then it's more of a matter of identifying the users and the requirements (you do seem to doubt that anyone but 'lawyers' care).
If you agree, then we can go on to see if we can agree on the requirements for each of those which might have an impact on the normative reference rules.

> What does stable mean to you? I think you are thinking of a "static" draft, not a
> stable one. Because a static dated draft cannot possibly be more stable that a
> draft that has been updated to fix an issue or clarify something.

I accept your distinction between static and stable, but not your definition of stable. The problem is the word "fix". You know when you have written some software and someone else "fixes" an "issue" they have with your code, but they really are screwing it up because they don't understand how you thought it was supposed to work? Except from their point of view, you were wrong and don't understand the new requirements?    We've seen ample cases where there is some question whether a "fix" of an issue in a specification is controversial. "That's not a bug, it's a feature!" applies to specs as well as code.  (Otherwise the Editor wouldn't WONTFIX anything.)


Do you agree? Do you think there are no such controversies?

W3C and other standards groups distinguish the categories of changes allowed by segmenting "editions" (only non-technical changes like clarifying text) and "versions".
Now I think you're using "stable" in some other way, because you are including (I believe) changes that include new features and technical changes you think might not be controversial enough to warrant discussion.


I said
> > But generally a SDO review process also:
> >
> > * Insures the specs have been reviewed to insure they work for more uncommon use cases and not just the "main" ones.
> > For example, while the primary purpose of HTML is for browser use, HTML is  used extensively in email, but not all HTML will work
> > in an email context.
> > * Insure the specs have been reviewed for security, privacy, performance,  internationalization, accessibility, even when
> > these qualities are not high priority to the primary implementors.
> > * Insures that patented proprietary technology is not imported into the platform inadvertently or without notice.
> > * Provides the ability to create stable test suites to test implementation conformance and interoperability.

and you replied:

> These are not given - but it's good that they do tend to happen at the W3C.

* The current way in which these things get "insured" in W3C requires "static" and "stable" documents to give to reviewers who are not prepared to follow updates.
* I wasn't convinced that the "Living Standard" review process accomplishes those values as effectively, but for now I'll accept the hypothesis that it can.

> > and indirectly, provides an audited and documented review process for other
> SDOs to evaluate.

W3C has documented processes. All I can find in WHATWG land is
http://wiki.whatwg.org/wiki/FAQ#What_does_.22Living_Standard.22_mean.3F

which offers little assurances other than " we are extremely careful!" If I have a non-browser implementation, what reassurance is it that new features that make unnecessary difficulties won't be introduced? You can only be "extremely careful" if you know my use cases and how they're implemented.

> > I think if those goals aren't high priority for you, you probably shouldn't
> bother with an SDO.
> You can still have those things as a priority, and the SDO can still fail to provide
> those things (or others can provide them). I'm not saying the W3C doesn't
> provide those things, just that it's like asking "why do you hate freedom?".

I agree that the likelihood of accomplishing the above goals for specification review is not really liked to whether the organization hosting the specification review process is *called* an SDO.   The organization either accomplishes the goal or not. There's either a good security review or there isn't.

But just as finance discovered that independent audits help control fraud, having documented and observed processes help control undue-influence in the standards process. (If you can stack the committee you can control the technology that wins.)    To call for audits is not the same as accusing anyone of fraud.

I'm not asking "why do you hate freedom?" -- that's an irrelevant analogy. (I'm not asking "why do you hate lawyers?" either)

> > "Living Standard" seems like an oxymoron -- fundamentally a "standard"
> which can change out from under you is no standard at all.
>
> I'm sorry, but it seems you don't have a good grasp of what a living standard is.

I likely don't, since I can find little written on the process other than the FAQ, which, as I noted, is completely unspecific. You've given some clues about independent review which I've found nowhere else.

> They don't just "change out from under you" - or no more than any other draft
> on /TR/.

This is an irrelevant comparison. I'm talking about "Living Standard" as meaning "Standard" in the same sense that Recommendations are, but /TR/ includes many kinds of documents, not just Recommendations.

> The goals of snapshotted and living specs are the same: get
> implemented and be interoperable.

I disagree they are "the same" even though they share some goals. While getting implemented and interoperable are goals of both, their goals are not the "same", because living specs do not share the goal of helping provide the evidence that the edition an implementor is implementing has survived a review for security, privacy, internationalization, accessibility, and other requirements that are not necessary to accomplish implementing them or making them interoperable.   That is because-- as has been pointed out -- the reviewers for those qualities are not dedicated resources.

.> Neither can achieve that is they can
> "change out from under you", now can they?

A "Recommendation" cannot change without substantially more review than a "Living Standard".  In IETF, an RFC is static, and a Standard cannot change without a very broad review either. A "Living Standard" can change with no more review than an Editor's Draft in a W3C working group, or (apparently) a single other reviewer,w at least in well-run groups with cooperative editors.

Do you disagree?

> So it's pretty silly to throw that
> "change out from under you" FUD around about about living standards. Please
> stop doing that.

Please don't call my arguments "silly". And "FUD" seems to be "generally a strategic attempt to influence perception by disseminating negative and dubious or false information."
impugns my motives, that I'm merely trying to inject "fear, uncertainty and doubt" into an otherwise clearly better alternative.  I'm comparing review processes.


> > The "standard" serves as a stable reference point for multiple parties to
> agree across the world, but it requires the implementors
> > and independent asynchronous review of the SAME spec.

> No it doesn't. The WHATWG's HTML specification trivially proves that.

I think the referent of "it" in my statement was unclear, so you were right to disagree.

I'm not sure what the WHATWG HTML trivially proves, but it doesn't prove to me
anything about assurance of adequate review for security, privacy, accessibility,
internationalization review.

> > If spec A refers to spec B normatively using "latest version" reference (rather
> than a specific dated reference), and B is an editor's draft,
> > then any attempt to stabilize A via the goals above will be thwarted, unless
> the destination of B is taken to be not
> > the "latest version", but rather the "latest version" that has also survived a
> review not only of B, but of the reference
> > from A to B.
>
> So you want B to be a broken and out-of-date spec? nice.

Sarcasm misplaced, and a straight reply would be more useful.

No, I don't think reviewers should be unaware of newer versions and editions to specifications, even when the "normative" reference is to a specific version or edition. Reviewers should have access to both, and should be specifically requested to review whether the update is significant or has an impact on the review.

> I think most implementers prefer B to be a spec that is maintained, up-to-date,
> and _actually stable_.

I think it depends on the nature of review, whether the quality being reviewed can be evaluated locally, and the choice of the meaning of "stable".

Let's take a case, the http://url.spec.whatwg.org Living Standard. My belief is that few if any implementors of URL generating or parsing libraries (outside of the browser community) are reviewing changes to that document before they are published.  Mainly because that specification is written in such an impenetrable style as to make it near impossible to perceive the simplest attributes about the format that the parser describes accepts.

And further they are unlikely to be willing to review a document changing as rapidly as the Living Standard.

So I don't think it would be fair to call that document "stable".

Similarly for http://mimesniff.spec.whatwg.org/ -- I don't think that document is "stable" either, as many of the bugs I've reported on that document have not yet been "fixed".

> That is, stable: that is constantly being fixed and maintained to be at it's highest quality at any period of time.


 Neither of those documents are at their 'highest quality" in my opinion.

I really don't understand this use of the word "stable". Software doesn't break down like a motor subject to friction and heat. Whether something is "fixed" or "maintained" or "highest quality" is often debatable, and whether a change is a "fix" depends on your requirements.

>  From that point of view, it is unhelpful to point to a dated draft (dated drafts are by definition "dated").

I still don't see any point of view where knowing the dated draft the original normative reference was intended for and also the version history of any more recent (proposed or accepted) editions is unhelpful.


> > There may be other ways of handling the dependencies that would be more complicated, but might accomplish the goals of simultaneously allowing
 referencing the 'latest version' but also being explicit about which versions reviewers were expected to read at the time of review.
>
> Why? that seems like a waste of time for the reviewers.

Reviewers do have to allocate how they spend their time, but giving them more information in a trivial way ... given the copious amounts of other information available ... wouldn't seem to increase the reviewer workload significantly.


> If reviewer A finds a
> bug, and it gets fixed - why should reviewer B be subjected to finding the same
> bug when it's already fixed? That's a good way to annoy your reviewers.

The problem is that the "living standards" include changes that are not agreed to be "fixes" for "bugs".
There are different kinds of review, which will determine which edition makes sense for you to review.

> >
> > Right now, we use the document status and location and filename to indicate
> and understand its review and approval status.
>
> Some people do, yes - but not everyone (specially people that need to read
> these specs every day for work). A lot of people just ignore that and go straight
> to latest because they know that they are *more stable* than stale dated
> ones.

The fact that many people find C useful does not detract from the utility of A and B. And the subject of normative references, the proposed new rules are only concerned with references FROM documents which purport to be "stable" (using my definition of stable).

> > There are a variety of ways the specifications could be arranged to keep the
> value of stable references while also giving implementors of the specifications
> a clear lead on where things are likely to go by pointing not only to the stable,
> reviewed version but also to the latest version.
>
> Ah! I see where your thinking is different to mine: you think that specs are only
> periodically reviewed! No no no, that view is totally wrong. As you mentioned,
> we use better open source tools now to allow our fellow humans to track
> changes (i.e., no more crappy CVS, less email, etc.).

Yes, some people review some specs continuously, most review other specs periodically. It's a resource issue. The core design group meets or talks daily, and specialists only review periodically.


> In the groups I'm involved with, no change can be made to a spec without
> having that change reviewed by another WG member or the public before it
> gets integrated into the spec. That's the magic of forcing everyone to use a Pull
> Request on GitHub. No change is integrated without a thorough independent
> review - no matter how small the change is!

You don't seem to distinguish between one other WG member review, and the kind of review that happens with a W3C working group consensus.

A thorough independent review by one other implementor might be approved by someone who is only familiar with the similar use cases and implementations.
It's great that happens, of course.


> > Or perhaps the "last call" of a core platform spec should be accompanied by a
> "last call" version specification which would indicate both the latest version
> pointer and also a dated version, for all normative references, of the version
> reviewed at the time of "last call".
>
> Has this ever actually been a real issue?

It's an issue that http://www.w3.org/TR/widgets/#sniff has a normative reference to "Media Type Sniffing" A. Barth and I. Hickson. IETF (Work in Progress).
And there were other use cases which led to the TAG issue.

 > If normative references change between last call and time of publication, it
> requires some judgement of whether last call needs to be repeated, or if the
> updates to referenced specifications are innocuous enough not to invalidate
> the previous "last call".
>
> I could be wrong, but I don't remember any lawyers complaining about
> HTML5's LC references pointing to Editor's drafts?

I don't know where "lawyers" get invoked; perhaps an attempt at humorously trivializing the question by suggesting that only lawyers care.

> http://www.w3.org/TR/2011/WD-html5-20110525/references.html#references

> Or for any other spec for that matter. If it ain't broke, don't fix it.

Some think it's "broke" and want to "fix" it. You think it isn't broken and the fixes proposed are counter-productive. Just the kind of controversy I say happens with any spec.

> The benefits
> of pointing to Editors drafts, living docs, or latest drafts far outweigh the
> benefits of pointing to dated drafts. That's easily demonstrable.

I agree that not giving anyone tools for discovering both approved and also proposed updates would be bad. I also think pointing to work that has been abandoned or has not reached a point of accuracy is a problem.
I'd like to see a way of annotating links with commentary on the appropriateness of various editions of the targets.

Received on Saturday, 20 April 2013 07:47:55 UTC