Re: suggestions for examples and explication wrt ABNF and header fields in draft-ietf-httpbis-p1

=JeffH wrote:
>  > RFC2616 said:
>  >
>  > "The grammar described by this specification is word-based. Except where
>  > noted otherwise, linear white space (LWS) can be included between any
>  > two adjacent words (token or quoted-string), and between adjacent words
>  > and separators, without changing the interpretation of a field." --
>  > <http://greenbytes.de/tech/webdav/rfc2616.html#basic.rules>
> 
> yes, good catch.
> 
> 
>  > So what got lost is the explanation word = token / quoted-string.
>  > Apparently we need to resurrect that part.
> 
> If you want to retain those two instances of using "word" rather than 
> "token", then yes, I agree you'd want to resurrect (in some way) that 

Well, a quoted-string isn't a token, so we can't say "token".

> portion of 2616 you quoted. I think you'd want to be sure include the 
> parenthetical "..two adjacent words (token or quoted-string).." that 
> indicates that what the prose is referring to as a "word" is either a 
> token or a quoted-string in the ABNF.

I just checked: "word" is used twice in Part 1, meaning 
token/quoted-string. It doesn't seem to be used in other parts, though.

That being said: I think the term it's useful to keep the term, even if 
only for consistency with previous specs.

So how about saying:

"Many HTTP/1.1 header field values consist of words (token or 
quoted-string) separated by whitespace or special characters. These 
special characters MUST be in a quoted string to be used within a 
parameter value (as defined in Section 6.2)."

(inserting "(token or quoted-string)")

That will also make it clear why the ABNF below actually defines "token" 
and "quoted-string"...

Checking the accumulated ABNF, we have...

accept-ext = OWS ";" OWS token [ "=" ( token / quoted-string ) ]
cache-extension = token [ "=" ( token / quoted-string ) ]
chunk-ext-val = token / quoted-str-nf
disp-extension-parm = token "=" ( token / quoted-string )
expect-params = ";" token [ "=" ( token / quoted-string ) ]
expectation-extension = token [ "=" ( token / quoted-string ) 
*expect-params ]
extension-pragma = token [ "=" ( token / quoted-string ) ]
te-ext = OWS ";" OWS token [ "=" ( token / quoted-string ) ]
value = token / quoted-string

...9 instances of "token / quoted-string", so maybe we should even add

   word = token / quoted-string

to the ABNF and use it throughout?

Best regards, Julian