On clarification text in specifications

The goal of writing protocol specifications is to create an on-demand
communicative experience conveying sufficient knowledge about the data
models, behavior, and on-the-wire marshalling of protocol elements such that
an implementor can create a protocol implementation that is correct with
respect to the specification, and is interoperable with other
implementations. 

To the best of my knowledge, there is no research on the effectiveness of
RFCs at communicating a protocol specification. As a result, there is simply
no factual basis for either side of the argument concerning adding, or not
adding, clarification text. The largely anecdotal evidence supports both
sides. RFCs are long, and generally getting longer, leading to more shallow
review. As a result, we should keep them as short as possible, with little
or no clarification text. On the other side, even the most successful
protocols have interoperability issues. It is my understanding that upwards
of 25% of the code base of FTP protocol libraries is dedicated to handling
interoperability issues with specific implementations. The IETF process
itself has been developed assuming that interoperability problems are
inevitable in implementations of RFCs, and need to be worked out over time.

Arguments from first principles can also support both sides. Given a
protocol that provides a clear description of its data model, behavior, and
on-the-wire marshalling, an implementor should be possible to use logical
implication to determine protocol behavior in all circumstances. Hence, no
clarification text should be necessary, since in the extreme one would list
all possible implications. Even when backing off from the extreme, it is
difficult to pick a criteria for which implications to explicitly include,
and which ones to omit. 

At present our specifications are only partially written using formal
languages, and there are no explicitly defined logical implication rules for
operating on specification assertions. As a result, it is *not* reasonable
to assume that even careful readers will be using the same set of
implication rules to answer questions about the specification. In fact,
given the lack of formally expressed implication rules, there is no way to
determine if different implication sets would lead to conflicting
conclusions about the specification. In this situation, the best proxy for
such knowledge is a situation where experienced protocol specifiers draw
different conclusions about protocol behavior. Since these experts have some
form of interal mechanism for making implications from protocol assertions,
lack of agreement is indicative of a situation where the application of
different implication rule sets lead to contradictory conclusions in the
specification. In such situations, it is useful to add clarification text to
disambiguate the outcomes of applying contradictory implication chains.

Since protocol specifiers can reasonably have differing opinions on the
value of clarifying text, what is a reasonable rule of thumb working groups
can use to determine when to include clarification text?

Ted Hardie's guidance here is right-on. If you have a situation where:

1) Expert opinion in a working group is divided after significant
discussion, or 
2) It took expert opinion considerable effort to develop a consensus
understanding of a non-normative behavior,

... then ... 

*) If adding clarification text seems likely to increase interoperability,
it should be included. If it seems likely to have no effect, or decrease
interoperability, omit it. Additionally, any added text should be judged to
be worth the cost in additional specification length.

That is, if you have reasonable evidence supporting the conclusion there is
an implication chain conflict among experienced readers, then if you can
develop clear language clarifying the issue, it should be included. Since
many implications are straightforward, and do not lead to disagreement, this
criteria acts to discourage widespread inclusion of clarification text.
Furthermore, any addition should weight its cost in terms of adding
specification length. A multi-page example to settle a minor point is
probably not a good tradeoff. One or two sentences to clarify a point is
likely a reasonable tradeoff.

Finally, it should be recalled that interoperability problems are
fantastically expensive. Changing a single interoperability problem in the
field can take years, and require significant engineering time across
multiple organizations. If we as protocol specifiers can eliminate such
errors, we save other engineers much time and money, and save end users from
frustration and delay.

- Jim

Received on Monday, 31 January 2005 17:30:51 UTC