An experiment in producing RFC1122-style requirements checklist

As Larry Masinter has reported, a small group of us has been holding
weekly teleconferences to hash out the remaining issues with the
HTTP/1.1 specification.  One general issue is that of specification
clarity, coupled with an editorial issue that we need to be sure
that all our MUSTs, SHOULDs, and MAYs are done right.

(Actually, the issue originally raised was one of specification
brevity, but my belief is that the people who want a briefer
specification are really asking for one that is easier to understand,
and I'm not convinced that simply shortening the current spec will
lead to understandability without ambiguity.)

Anyway, I suggested that we follow the lead of RFC1122/RFC1123
(the "Host Requirements" documents), and produce a checklist that
summarizes the MUST/SHOULD/MAY requirements of the document.  This
has several functions:
	(1) it should make it easier for developers to know what
	they have to do to comply with the final specification.
	(2) it helps us (the editors) be sure that we have tracked
	down all of the normative parts of the spec (i.e., those
	that directly affect claims of compliance).
	(3) it helps make clear whether a change to the text of
	the specification results in a change to what is required
	(i.e., a new/changed/deleted MUST/SHOULD/MAY) or is simply
	a clarification of the explanatory language.

This proposal of mine raised several questions:
	(A) is this kind of checklist really useful?
	(B) how much work would it take to produce it?
	(C) how many instances of MUST/SHOULD/MAY are wrong or missing?

So I volunteered to take a short section of RFC2068 and produce
a prototype checklist.  THIS IS NOT A FINAL PART OF THE SPECIFICATION.
The purpose of this exercise was to see whether doing a checklist
makes sense, NOT TO GENERATE A "CORRECT" CHECKLIST.  Therefore,
I will simply ignore all criticism of the actual contents of this
checklist.  I did try to make it as correct as possible, but I've
also included one intentially bogus item, just to confound anyone
who wants to nit-pick (and who doesn't bother to read this warning!)

Since the next two weeks on my schedule are already totally overloaded,
I arbitrarily allocated 30 minutes for this experiment (not including
composing this message).  Even in that short time, I was able to
get through the first 17 pages (or so) of section 13 of RFC2068.

The table below follows the modification of the RFC1122 format.  I.e.,
rather than follow that format exactly, I made separate columns for
Server, Proxy, and Client requirements.  It may be that what we really
want are columns for Server, Client, and Cache (or more than three
columns); some of this could require some fine-tuning.  I also
discovered a few places where "musts" appear in lower case, or only
implicitly, but the point of this experiment is not to worry about the
specific normative issues; I was just trying to get a feel for the
table format.  However, the answer to question (C) above seems to
be "about 10%-20%", which implies that we certainly need to review
the MUST/SHOULD/MAY language before finishing the next draft of the
spec.

Anyway, based on taking 30 minutes for 17 pages, it should take
about 5 person-hours of work to do the entire 162-page document.
Although I'm sure my concentration would have dropped off after
doing much more of this.  Therefore, the answer to question
(B) above is probably "not a tremendous amount of work".

This leaves question (A): is this useful?  We invite members of
the working group to comment, succinctly, on the utility of this
kind of table (as part of a future edition of the HTTP/1.1 spec),
and on the format of the table (i.e., did I get the columns right?)

Note that some of the caching requirements are stated in section
13, and others are stated in sections 14 (on specific headers)
or 9 (Methods) or 10 (Status codes).  I'm assuming that implementors
will be able to deal with a checklist that isn't carefully organized
into topic areas, but I suppose there is no real reason why this
has to be done in strict section-number order (which is what I've
done so far).  Comments on whether the table should be in strict
section-number order are welcome, although the alternative might
lead to a lot more work for the editors (i.e., we might not feel
like spending the time).

-Jeff

RFC2068 Section 13: HTTP Caching (through section 13.4)

DRAFT DRAFT DRAFT DRAFT DRAFT DRAFT DRAFT DRAFT DRAFT DRAFT DRAFT DRAFT DRAFT

Feature	summary				Section	Server	Proxy	Client	Note

Meets cache correctness rules 1-4	13.1.1	na	MUST	MUST	1
If server unreachable, follow rules 1-4	13.1.1	na	SHOULD	SHOULD	1
 " ", or return error/warning		13.1.1	na	MUST	MUST	1
Forward initially-stale response	13.1.1	na	SHOULD	SHOULD	1
Don't revalidate initially-stale resp	13.1.1	na	SH NOT	SH NOT	1
Display warning on received-stale resp	13.1.1	na	na	MAY

Attach Warning to stale response	13.1.5	na	MUST	na	
Obey client request for freshness	13.1.5	na	SHOULD	na
Revalidate init-stale cached response	13.2.1	na	SHOULD	SHOULD	1

Comply with Age calculation algorithm	13.2.3	na	MUST	MUST	1, 99
Interpret Age rel. to initiation time	13.2.3	na	MUST	MUST	1

max-age overrules Expires		13.2.4	na	MUST	MUST	1, 99
Use heuristic if no Expires or max-age	13.2.4	na	MAY	MAY	1
Warn if heuristic & Age > 24 hours	13.2.4	na	MUST	MUST	1, 99
Base heuristic on Last-Mod time		13.2.4	na	SHOULD	SHOULD	1
Freshness test based on age		13.2.4	na	MUST	MUST	1, 99

Ignore new response with older Date	13.2.5	na	MAY	MAY	1
Use response with most recent Date	13.2.5	na	MUST	MUST	1
Re-revalidate when new resp seems old	13.2.6	na	SHOULD	SHOULD	1
Don't expect disambiguation for = Dates	13.2.6	MST NT	na	na

Weak comparison for non-subrange reqs	13.3.3	MAY	MAY	na
Strong comparison for subrange/non-GET	13.3.3	MUST	MUST	na
Last-modified implicitly weak		13.3.3	MUST	MUST	MUST	99

Strong etag changes always		13.3.4	MUST	na	na
Weak etag changes when "significant"	13.3.4	SHOULD	na	na
Use etag in conditionals, if available	13.3.4	na	MUST	MUST
Use Last-Modified, if avail & no etag	13.3.4	na	SHOULD	SHOULD
 " ", with subrange reqs		13.3.4	na	MAY	MAY
Disable use of Last-Mod w/subranges	13.3.4	na	na	SHOULD	99
Use both Last-Mod & etag, if avail	13.3.4	na	SHOULD	SHOULD
Use most restrictive validator		13.3.4	na	MUST	MUST	1

Validate only w/etag or Last-Modified	13.3.5	MUST	MUST	MUST	98

Responses normally cachable		13.4	na	MAY	MAY	99
HTTP/1.0 responses not cachable		13.4	na	MST NT	MST NT	1
No validator/expires/max-age ->no cache	13.4	na	SHOULD	SHOULD	99
 " " unless unusual situation		13.4	na	MAY	MAY	99
Cachable status codes are
	200, 203, 206, 300, 301, 410	13.4	na	MAY	MAY	99
Cache 206 only if ranges supported	13.4	na	MST NT	MST NT	1
Other status codes w/out explict perm	13.4	na	MST NT	MST NT	1

   -------
Footnotes:
1) Cache rule applies to client-local caches as well as proxy caches.
98) No explicit upper-case conformance requirement stated in RFC2068,
	perhaps not actually a conformance requirement.
99) No explicit upper-case conformance requirement stated in RFC2068

DRAFT DRAFT DRAFT DRAFT DRAFT DRAFT DRAFT DRAFT DRAFT DRAFT DRAFT DRAFT DRAFT

Received on Thursday, 29 May 1997 11:05:23 UTC