Re: New resource: Normative References to W3C Standards

I applaud Larry for recognizing that there are different communities with
different needs in play here.

I would add that we should remember that many of the choices made about web
development are cultural and/or personal and not necessarily technical. Web
developers are human beings, with cliques, fashion trends, etc.

When possible, the W3C I think should try to satisfy a wide-range of web
concerns. One way to enhance this is to make sure multi-versioned specs
have both a latest version url, as well as a url for a particular version.
In addition, as a note next to the normative specifications links you could
have a link for the latest version of the relevant technology.

With non-versioned specs, it might be best to have a system in place for
taking an archived snapshot of the spec that you are referencing, possibly
adding notes (which should be distinguishable from the document), about
which parts of the spec are unstable or unreliable. Ideally, this should
include a built in button for doing a diff with the latest version of the
non-versioned standard.

One important reason for this is that specs move at different paces, and
some specs become effectively abandoned, even if they reference a "living"
standard. Thus Spec A refers to living standard B, living standard B makes
a correction to a concept that it is used in Spec A. It is important that
an implementer be able to tell that Spec A is possibly flawed because it is
out of date in an important way. On the other hand, the difference between
the version of standard B that Spec A uses and the version of standard B
which is the latest might be trivial or not relevant to the implementer's
use of Spec A.

For example, the WebIDL spec has changed numerous times over the last few
years in ways that have forced spec writers to revise the WebIDL text they
are using, if someone writes a WebIDL parser they should have some way to
see that the spec whose WebIDL they are trying to parse is incompatible
with the definition of WebIDL they are using, and in what ways it is
incompatible.

My authority here is limited, I was a web developer, now work for a tool
vendor in the embedded/safety-critical space (whose official positions I am
not representing in any way). I do still work with a lot of web-based tools
but mostly not on the open web (ie, tools like Jenkins, html reports from
my company's software, etc.), but if you like my 2 cents you may have it.

With regards,
John Thomas



On Sat, Apr 20, 2013 at 3:47 AM, Larry Masinter <masinter@adobe.com> wrote:

> I want to reply to an earlier message in the thread, to try to keep the
> topic focused on the one use case normative references.
> The TAG took up the issue in 2009
> http://lists.w3.org/Archives/Public/www-tag/2009Oct/0075.html
> and later I took it as an action to develop an update
> https://www.w3.org/2001/tag/group/track/actions/350
> which was  resolved it by pointing to a previous W3C recommendation
> http://www.w3.org/TR/qaframe-spec/#ref-define-practice
>
> based on multiple discussions. Perhaps updating
> http://www.w3.org/wiki/NormativeReferences would be useful.
>
> I think this is an important question to resolve in the W3C.
>
> I think you are using the term "stable" in a very different way than I do,
> and I claim than "most" do.  I'm willing to distinguish:
>
>  - static specifications (document doesn't change)
>  -  stable specifications (only non-technical editorial changes)
>  - "latest version" specifications,
>
> and would like to assert only that  there are different user communities
> who might have different needs for these,
> and asks that we focus on  the main question whether there are hard, soft,
> or no rules for normative references among the categories.
>
> I'm not sure everyone agrees with this formulation of the problem, so
> please speak up whether or not you agree.
> If you don't, then it's more of a matter of identifying the users and the
> requirements (you do seem to doubt that anyone but 'lawyers' care).
> If you agree, then we can go on to see if we can agree on the requirements
> for each of those which might have an impact on the normative reference
> rules.
>
> > What does stable mean to you? I think you are thinking of a "static"
> draft, not a
> > stable one. Because a static dated draft cannot possibly be more stable
> that a
> > draft that has been updated to fix an issue or clarify something.
>
> I accept your distinction between static and stable, but not your
> definition of stable. The problem is the word "fix". You know when you have
> written some software and someone else "fixes" an "issue" they have with
> your code, but they really are screwing it up because they don't understand
> how you thought it was supposed to work? Except from their point of view,
> you were wrong and don't understand the new requirements?    We've seen
> ample cases where there is some question whether a "fix" of an issue in a
> specification is controversial. "That's not a bug, it's a feature!" applies
> to specs as well as code.  (Otherwise the Editor wouldn't WONTFIX anything.)
>
>
> Do you agree? Do you think there are no such controversies?
>
> W3C and other standards groups distinguish the categories of changes
> allowed by segmenting "editions" (only non-technical changes like
> clarifying text) and "versions".
> Now I think you're using "stable" in some other way, because you are
> including (I believe) changes that include new features and technical
> changes you think might not be controversial enough to warrant discussion.
>
>
> I said
> > > But generally a SDO review process also:
> > >
> > > * Insures the specs have been reviewed to insure they work for more
> uncommon use cases and not just the "main" ones.
> > > For example, while the primary purpose of HTML is for browser use,
> HTML is  used extensively in email, but not all HTML will work
> > > in an email context.
> > > * Insure the specs have been reviewed for security, privacy,
> performance,  internationalization, accessibility, even when
> > > these qualities are not high priority to the primary implementors.
> > > * Insures that patented proprietary technology is not imported into
> the platform inadvertently or without notice.
> > > * Provides the ability to create stable test suites to test
> implementation conformance and interoperability.
>
> and you replied:
>
> > These are not given - but it's good that they do tend to happen at the
> W3C.
>
> * The current way in which these things get "insured" in W3C requires
> "static" and "stable" documents to give to reviewers who are not prepared
> to follow updates.
> * I wasn't convinced that the "Living Standard" review process
> accomplishes those values as effectively, but for now I'll accept the
> hypothesis that it can.
>
> > > and indirectly, provides an audited and documented review process for
> other
> > SDOs to evaluate.
>
> W3C has documented processes. All I can find in WHATWG land is
> http://wiki.whatwg.org/wiki/FAQ#What_does_.22Living_Standard.22_mean.3F
> which offers little assurances other than " we are extremely careful!" If
> I have a non-browser implementation, what reassurance is it that new
> features that make unnecessary difficulties won't be introduced? You can
> only be "extremely careful" if you know my use cases and how they're
> implemented.
>
> > > I think if those goals aren't high priority for you, you probably
> shouldn't
> > bother with an SDO.
> > You can still have those things as a priority, and the SDO can still
> fail to provide
> > those things (or others can provide them). I'm not saying the W3C doesn't
> > provide those things, just that it's like asking "why do you hate
> freedom?".
>
> I agree that the likelihood of accomplishing the above goals for
> specification review is not really liked to whether the organization
> hosting the specification review process is *called* an SDO.   The
> organization either accomplishes the goal or not. There's either a good
> security review or there isn't.
>
> But just as finance discovered that independent audits help control fraud,
> having documented and observed processes help control undue-influence in
> the standards process. (If you can stack the committee you can control the
> technology that wins.)    To call for audits is not the same as accusing
> anyone of fraud.
>
> I'm not asking "why do you hate freedom?" -- that's an irrelevant analogy.
> (I'm not asking "why do you hate lawyers?" either)
>
> > > "Living Standard" seems like an oxymoron -- fundamentally a "standard"
> > which can change out from under you is no standard at all.
> >
> > I'm sorry, but it seems you don't have a good grasp of what a living
> standard is.
>
> I likely don't, since I can find little written on the process other than
> the FAQ, which, as I noted, is completely unspecific. You've given some
> clues about independent review which I've found nowhere else.
>
> > They don't just "change out from under you" - or no more than any other
> draft
> > on /TR/.
>
> This is an irrelevant comparison. I'm talking about "Living Standard" as
> meaning "Standard" in the same sense that Recommendations are, but /TR/
> includes many kinds of documents, not just Recommendations.
>
> > The goals of snapshotted and living specs are the same: get
> > implemented and be interoperable.
>
> I disagree they are "the same" even though they share some goals. While
> getting implemented and interoperable are goals of both, their goals are
> not the "same", because living specs do not share the goal of helping
> provide the evidence that the edition an implementor is implementing has
> survived a review for security, privacy, internationalization,
> accessibility, and other requirements that are not necessary to accomplish
> implementing them or making them interoperable.   That is because-- as has
> been pointed out -- the reviewers for those qualities are not dedicated
> resources.
>
> .> Neither can achieve that is they can
> > "change out from under you", now can they?
>
> A "Recommendation" cannot change without substantially more review than a
> "Living Standard".  In IETF, an RFC is static, and a Standard cannot change
> without a very broad review either. A "Living Standard" can change with no
> more review than an Editor's Draft in a W3C working group, or (apparently)
> a single other reviewer,w at least in well-run groups with cooperative
> editors.
>
> Do you disagree?
>
> > So it's pretty silly to throw that
> > "change out from under you" FUD around about about living standards.
> Please
> > stop doing that.
>
> Please don't call my arguments "silly". And "FUD" seems to be "generally a
> strategic attempt to influence perception by disseminating negative and
> dubious or false information."
> impugns my motives, that I'm merely trying to inject "fear, uncertainty
> and doubt" into an otherwise clearly better alternative.  I'm comparing
> review processes.
>
>
> > > The "standard" serves as a stable reference point for multiple parties
> to
> > agree across the world, but it requires the implementors
> > > and independent asynchronous review of the SAME spec.
>
> > No it doesn't. The WHATWG's HTML specification trivially proves that.
>
> I think the referent of "it" in my statement was unclear, so you were
> right to disagree.
>
> I'm not sure what the WHATWG HTML trivially proves, but it doesn't prove
> to me
> anything about assurance of adequate review for security, privacy,
> accessibility,
> internationalization review.
>
> > > If spec A refers to spec B normatively using "latest version"
> reference (rather
> > than a specific dated reference), and B is an editor's draft,
> > > then any attempt to stabilize A via the goals above will be thwarted,
> unless
> > the destination of B is taken to be not
> > > the "latest version", but rather the "latest version" that has also
> survived a
> > review not only of B, but of the reference
> > > from A to B.
> >
> > So you want B to be a broken and out-of-date spec? nice.
>
> Sarcasm misplaced, and a straight reply would be more useful.
>
> No, I don't think reviewers should be unaware of newer versions and
> editions to specifications, even when the "normative" reference is to a
> specific version or edition. Reviewers should have access to both, and
> should be specifically requested to review whether the update is
> significant or has an impact on the review.
>
> > I think most implementers prefer B to be a spec that is maintained,
> up-to-date,
> > and _actually stable_.
>
> I think it depends on the nature of review, whether the quality being
> reviewed can be evaluated locally, and the choice of the meaning of
> "stable".
>
> Let's take a case, the http://url.spec.whatwg.org Living Standard. My
> belief is that few if any implementors of URL generating or parsing
> libraries (outside of the browser community) are reviewing changes to that
> document before they are published.  Mainly because that specification is
> written in such an impenetrable style as to make it near impossible to
> perceive the simplest attributes about the format that the parser describes
> accepts.
>
> And further they are unlikely to be willing to review a document changing
> as rapidly as the Living Standard.
>
> So I don't think it would be fair to call that document "stable".
>
> Similarly for http://mimesniff.spec.whatwg.org/ -- I don't think that
> document is "stable" either, as many of the bugs I've reported on that
> document have not yet been "fixed".
>
> > That is, stable: that is constantly being fixed and maintained to be at
> it's highest quality at any period of time.
>
>
>  Neither of those documents are at their 'highest quality" in my opinion.
>
> I really don't understand this use of the word "stable". Software doesn't
> break down like a motor subject to friction and heat. Whether something is
> "fixed" or "maintained" or "highest quality" is often debatable, and
> whether a change is a "fix" depends on your requirements.
>
> >  From that point of view, it is unhelpful to point to a dated draft
> (dated drafts are by definition "dated").
>
> I still don't see any point of view where knowing the dated draft the
> original normative reference was intended for and also the version history
> of any more recent (proposed or accepted) editions is unhelpful.
>
>
> > > There may be other ways of handling the dependencies that would be
> more complicated, but might accomplish the goals of simultaneously allowing
>  referencing the 'latest version' but also being explicit about which
> versions reviewers were expected to read at the time of review.
> >
> > Why? that seems like a waste of time for the reviewers.
>
> Reviewers do have to allocate how they spend their time, but giving them
> more information in a trivial way ... given the copious amounts of other
> information available ... wouldn't seem to increase the reviewer workload
> significantly.
>
>
> > If reviewer A finds a
> > bug, and it gets fixed - why should reviewer B be subjected to finding
> the same
> > bug when it's already fixed? That's a good way to annoy your reviewers.
>
> The problem is that the "living standards" include changes that are not
> agreed to be "fixes" for "bugs".
> There are different kinds of review, which will determine which edition
> makes sense for you to review.
>
> > >
> > > Right now, we use the document status and location and filename to
> indicate
> > and understand its review and approval status.
> >
> > Some people do, yes - but not everyone (specially people that need to
> read
> > these specs every day for work). A lot of people just ignore that and go
> straight
> > to latest because they know that they are *more stable* than stale dated
> > ones.
>
> The fact that many people find C useful does not detract from the utility
> of A and B. And the subject of normative references, the proposed new rules
> are only concerned with references FROM documents which purport to be
> "stable" (using my definition of stable).
>
> > > There are a variety of ways the specifications could be arranged to
> keep the
> > value of stable references while also giving implementors of the
> specifications
> > a clear lead on where things are likely to go by pointing not only to
> the stable,
> > reviewed version but also to the latest version.
> >
> > Ah! I see where your thinking is different to mine: you think that specs
> are only
> > periodically reviewed! No no no, that view is totally wrong. As you
> mentioned,
> > we use better open source tools now to allow our fellow humans to track
> > changes (i.e., no more crappy CVS, less email, etc.).
>
> Yes, some people review some specs continuously, most review other specs
> periodically. It's a resource issue. The core design group meets or talks
> daily, and specialists only review periodically.
>
>
> > In the groups I'm involved with, no change can be made to a spec without
> > having that change reviewed by another WG member or the public before it
> > gets integrated into the spec. That's the magic of forcing everyone to
> use a Pull
> > Request on GitHub. No change is integrated without a thorough independent
> > review - no matter how small the change is!
>
> You don't seem to distinguish between one other WG member review, and the
> kind of review that happens with a W3C working group consensus.
>
> A thorough independent review by one other implementor might be approved
> by someone who is only familiar with the similar use cases and
> implementations.
> It's great that happens, of course.
>
>
> > > Or perhaps the "last call" of a core platform spec should be
> accompanied by a
> > "last call" version specification which would indicate both the latest
> version
> > pointer and also a dated version, for all normative references, of the
> version
> > reviewed at the time of "last call".
> >
> > Has this ever actually been a real issue?
>
> It's an issue that http://www.w3.org/TR/widgets/#sniff has a normative
> reference to "Media Type Sniffing" A. Barth and I. Hickson. IETF (Work in
> Progress).
> And there were other use cases which led to the TAG issue.
>
>  > If normative references change between last call and time of
> publication, it
> > requires some judgement of whether last call needs to be repeated, or if
> the
> > updates to referenced specifications are innocuous enough not to
> invalidate
> > the previous "last call".
> >
> > I could be wrong, but I don't remember any lawyers complaining about
> > HTML5's LC references pointing to Editor's drafts?
>
> I don't know where "lawyers" get invoked; perhaps an attempt at humorously
> trivializing the question by suggesting that only lawyers care.
>
> > http://www.w3.org/TR/2011/WD-html5-20110525/references.html#references
> > Or for any other spec for that matter. If it ain't broke, don't fix it.
>
> Some think it's "broke" and want to "fix" it. You think it isn't broken
> and the fixes proposed are counter-productive. Just the kind of controversy
> I say happens with any spec.
>
> > The benefits
> > of pointing to Editors drafts, living docs, or latest drafts far
> outweigh the
> > benefits of pointing to dated drafts. That's easily demonstrable.
>
> I agree that not giving anyone tools for discovering both approved and
> also proposed updates would be bad. I also think pointing to work that has
> been abandoned or has not reached a point of accuracy is a problem.
> I'd like to see a way of annotating links with commentary on the
> appropriateness of various editions of the targets.
>