#468 p2: Expectation extensions (was: Expect header 'understand' vs 'meet')

I spent a few days trying to wrangle the issues around the
Expect header field in p2 (#466, #467, #458, #468) and have
run into a brick wall.  I think we should reopen #468,
remove everything except 100-continue, and then specify the
100-continue mechanism as it is implemented in practice.

I tried to rewrite the section so that the existing text
would make sense as a must-understand-or-fail feature for
HTTP/1.1.  I was able to do that, based on the fantasy that
HTTP/1.1 servers had at least deployed that much.  Apparently,
I had already forgotten the testing I did two years ago, and
nothing has changed since then.

In short, Apache httpd is the only server that I could find
which sent 417 Expectation Failed in response to an expectation
extension.  Apache does not actually parse the syntax -- it
simply rejects anything that isn't 100-continue.  There exists
a patched version of nginx that does the same, but only for
file upload requests that are about to succeed.  IIS does not
appear to support Expect at all, though it may for some resources
that I could not find.

Then, I stopped writing, since there is no point in
specifying a must-understand feature that still hasn't been
implemented consistently 16 years after it was originally defined.

What we can do is define how "Expect: 100-continue" is implemented
in practice as an indication that the client will wait some
implementation-dependent amount of time to see if it gets an
error before continuing to send the request body.  We can specify
that sensibly, including its shortcomings, if we discard the
must-understand semantics and the protocol versioning bit
about responding with 417 if the next hop is HTTP/1.0.

I know that #468 was closed due to lack of interest, but the
alternative is to specify something that hasn't been implemented
and will never be interoperable outside a closed environment
(where both client and server are controlled by the same org),
and we don't need Expect extensions in a closed environment.

....Roy

 
On Sep 8, 2011, at 11:00 AM, Roy T. Fielding wrote:

> On Sep 8, 2011, at 4:43 AM, Amos Jeffries wrote:
> 
>> On 08/09/11 09:09, Roy T. Fielding wrote:
>>> On Sep 7, 2011, at 1:24 PM, Julian Reschke wrote:
>>> 
>>>> On 2011-09-07 22:18, Jan Algermissen wrote:
>>>>> ..
>>>>>> No, that's unfortunately correct. Maybe we need to mention it.
>>>>> 
>>>>> Sorry to keep bugging - do you know the rationale behind this?
>>>> 
>>>> That predates my interest in HTTP.
>>>> 
>>>> Roy?
>>> 
>>> "Expect" is not part of my original design.  It is in the archives, somewhere.
>>> 
>>> http://lists.w3.org/Archives/Public/ietf-http-wg-old/1997MayAug/0263.html
>>> http://lists.w3.org/Archives/Public/ietf-http-wg-old/1997MayAug/0296.html
>>> http://lists.w3.org/Archives/Public/ietf-http-wg-old/1997MayAug/0316.html
>>> http://lists.w3.org/Archives/Public/ietf-http-wg-old/1998MayAug/0165.html
>>> 
>>> IIRC, we had a discussion in the editorial group wherein I pointed
>>> out that the MUST respond to 417 on extensions was a fatal flaw --
>>> we might as well have made it non-extensible.
>>> 
>>> ....Roy
>> 
>> I don't think it is entirely fatal.
>> 
>> Expect: is cleanly restricted to probing features of the underlying pathway or requirements for success of the particular request+entity. Where things either work or fail hop-by-hop, no middle ground for almost worked, but room for middleware recovery.
>> 
>> Perhapse that usability scope needs to be re-worded and made clear.
> 
> Except for the minor detail that Expect was added after RFC2068, and hence
> anyone implementing to the older RFC (or anyone just failing to implement
> every single aspect of 2616) will not fail as defined.
> 
>> Here are some extensions where I believe using it could benefit HTTP implementations today:
>> 
>> 
>> * testing for next-hop pipeline support
>> 
>> As I already suggested in the pipeline thread. Sending "Expect: pipeline" or similar would allow intermediaries pipeline support to be tested and alert them to the fact that extra efforts may need to be made to keep that connection alive.
>> (I know the round-trip arguments, just pointing out it could be a useful extension of Expect: which is up to the client to decide whether to use or not.)
> 
> It is a performance extension.  We can't deploy a performance extension
> using a field that drastically kills performance.
> 
>> * testing for working NTLM auth
>> 
>> Sending "Expect: connection-auth" during the auth logins over the Internet could be used by browsers to boost NTLM/Negotiate reliability. Intermediaries who cannot support or don't understand the requirements for NTLM can send 417 and break the potential infinite loop of auth, lost credentials, connection re-use mistakes, etc, etc.
>> Yes the intermediaries can (and do) today detect the auth headers going through. This still breaks when transiting the intermediaries which lack proper support.
> 
> True.  OTOH, such a client would break on every working install
> of NTLM as well, which means it can't ever be deployed because
> the main reason to use NTLM is to support intranet services.
> 
>> * testing for secure communications
>> 
>> Sending "Expect: tls" during a request to indicate that it must only transit over encrypted connections. Intermediaries who don't comply or understand, send 417 and the connection fails to safety.
> 
> The client already controls whether or not it uses TLS.
> Expect cannot be used to force an intermediary to do something
> on the next hop, since the client can't trust the intermediary
> to respect Expect.
> 
>> I'm sure thats not the end of the possibilities. But it goes to show that Expect: is useful in its current form because of the MUST.
> 
> I don't think so.  Even the 100-continue support would have failed
> if Apache hadn't deployed it early.  Expect is only useful for that
> and for private extensions within corporate networks where one can
> ensure that all of the internal servers support Expect in the same
> way.
> 
> ....Roy

Received on Saturday, 10 August 2013 08:41:00 UTC