Re: [Fwd: I-D ACTION:draft-decroy-http-progress-00.txt]

Henrik Nordstrom wrote:
> ons 2007-02-14 klockan 11:41 +1300 skrev Adrien de Croy:
>>> Well.. no matter what the client only has the choice of either closing
>>> the connection or sending the request body. That's the only two
>>> available options once the request headers have been sent indicating
>>> there will be a request entity body.
>>>   
>> There is another option.  Waiting for a 100 Continue before sending 
>> message body.
>
> Well, what I am trying to say is that this does not change the available
> options for finishing this request.
>
OK

>> The aim of the defer status is to remove option 2.  So there is then 
>> ONLY option 3.  The client must wait
>> for a 100 continue (whilst still keeping the connection open) before 
>> sending the request body.
>
> You still have the same window where the client must guess, just
> hopefully a bit smaller.. 
understood.  If the proxy checks this first, it should be able to be a 
very small window.

If we went with a tag approach, where the client advertises support for 
flow control with e.g. a connection
tag, then it would be more deterministic.

Part of the spec would also need to mandate a minimum amount of time 
that the client MUST wait, e.g. 100ms.
before sending the body, rather than using RTT, which is often only 1 or 
2 ms for a connection to a proxy.

Or the client may already know that the proxy uses this method.

>
> And in case of 102 generated by a proxy you also run into the problem of
> how to guarantee there finally will be a 100 Continue delivered. And as
> result the proxy must also implement the timer and "fake" a 100 Continue
> if no 1xx is seen from the next-hop within a reasonable timeframe.
>
true. That's what I was also planning on having to do anyway.  The 
entity that sends a "102 wait" MUST
guarantee that a 100 continue will be sent to the client.  So the proxy 
will need to act as a client for
the next-hop server, which is what it needs to do anyway.  If the 
next-hop server doesn't support this
which it wont for some time even if we get the spec through, then the 
proxy needs to manage this.

>> So, if you have a client that honours a 1xx defer status.
>>
>> case 1, proxy requires auth, server does not.
>>
>> 1. Client sends request headers only (as per current RFC2616 
>> suggestions) and goes into a short wait state
>> 2. proxy sees that there will be a request body, so immediately sends 
>> "HTTP/1.1 1xx wait please"
>> 3. client terminates timer for when it would otherwise send request 
>> body.  It does not send request body
>> 4 proxy then sends "HTTP/1.1 407 auth required"
>
> * client closes the connection to avoid transmitting the request body

Why would it though?   there's no actual need to close here.  Just 
simply not send the request body,
rather than closing, and pop up an auth dialog box.  It makes no sense 
to waste resource by sending
something the client KNOWS will be dumped.  The Greens will be after us !

clients I've seen are really good at keeping the connection to a proxy 
open, even when connecting to
multiple different sites requiring different auth.

>
>> 5 client establishes auth credentials with proxy, still not sending 
>> request body6 proxy connects to upstream server. 
>> 7. server sends 100 continue
>> 8 proxy sends server response back through to client
>> 9 client sends request body
>> 10 proxy relays request body.
>> 11. server sends 200 ok.
>> 12. proxy relays 200 ok.
>> 13 done.
>>
>> case 2, proxy requires auth, and server also requires auth.
>>
>> 1. Client sends request headers only (as per current RFC2616 
>> suggestions) and goes into a short wait state
>> 2. proxy sees that there will be a request body, so immediately sends 
>> "HTTP/1.1 1xx wait please"
>> 3. client terminates timer for when it would otherwise send request 
>> body.  It does not send request body
>> 4 proxy then sends "HTTP/1.1 407 auth required"
>
> * client closes the connection to avoid transmitting the request body.
>
>> 5 client establishes auth credentials with proxy, still not sending 
>> request body
>> 6 proxy connects to upstream server. 
>> 7. server sends 401 auth required
>> 8 proxy sends server response back through to client
>> 9 client resends request with server credentials  Still no request body
>> 10 proxy relays request to server
>> 11. server sends 100 continue
>> 12. proxy relays 100 continue
>> 13. client sends request body
>> 14 proxy relays request body
>> 15 server sends 200 ok
>> 16 proxy relays 200 ok.
>> 17 done.
>>
>> case 3, only server requires auth
>>
>> 1. Client sends request headers only (as per current RFC2616 
>> suggestions) and goes into a short wait state
>> 2. proxy sees that there will be a request body, so immediately sends 
>> "HTTP/1.1 1xx wait please"
>> 3. client terminates timer for when it would otherwise send request 
>> body.  It does not send request body
>> 4 proxy connects to upstream server. 
>> 5. server sends 401 auth required
>> 6 proxy sends server response back through to client
>> 7 client resends request with server credentials  Still no request body
>> 8 proxy relays request to server
>> 9. server sends 100 continue
>> 10. proxy relays 100 continue
>> 11. client sends request body
>> 12 proxy relays request body
>> 13 server sends 200 ok
>> 14 proxy relays 200 ok.
>> 15 done.
>
>
> case 4, server does not implement 100 Continue
>
> 1. Client sends request headers only (as per current RFC2616
> suggestions) and goes into a short wait state
> 2. proxy sees that there will be a request body, so immediately sends "HTTP/1.1 1xx wait please"
> 3. client terminates timer for when it would otherwise send request body.  It does not send request body
> 4. proxy connects to upstream server. 
> 5. server sends 401 auth required
> 6. proxy sends server response back through to client
> 7*. client closes the connection to avoid transmitting the request body
> 8. client resends request headers with server credentials
> 8. proxy relays request headers to server
> 9. server doesn't implement 100 Continue and none gets sent..

If it's a new connection from step 8, then it's a new session.

if the client closes at step 7 above then it continues as

0. client makes new connection to proxy
1. Client sends request headers only including server credentials 
2. proxy sees that there will be a request body, so immediately sends "HTTP/1.1 1xx wait please"
3. client terminates timer for when it would otherwise send request body.  It does not send request body
4. proxy connects to upstream server. 
5. proxy sends client request including credentials to server (presumably the credentials are valid)
6. proxy waits for 100 continue from server.  If it doesn't receive one within some time, it sends 100 continue to client.
if it does get one from the server within that time, it sends it on to the client.
7. client sends request body.
8. proxy relays request body
9. server sends 200 OK
10 proxy relays server response
11 done.

clients that support NTLM don't close when they get a 401 or 407, but I guess they may if they haven't
sent a request body.  Part of this support in the client for a wait response would be changes to ensure
that the client did not close the connection, since that's no longer necessary, and breaks connection-based auth



>
>
>> All this can be done on the same connection between client and proxy, 
>> proxy and server.
>>
>> I should really draw up a flow chart of all this.  from the client's 
>> point of view, it sends a request, and if it receives a 102
>> wait please, it will then receive any number of:
>>
>> a. a 100 continue
>> b. an auth challenge
>> c. some failure code indicating the request is denied.
>> d. connection closed (some failure condition)..
>>
>> until it has received a 100 continue however, it won't send the request 
>> body.
>
> Yes, but you do understand that for the connection to be reused to
> complete authentication etc the first request must first be completed
> which requires either sending the request body or closing the
> connection.

OK, I see where you're coming from.  I was assuming we would change the 
client to not disconnect.

I should not assume :)

the request would have effectively been "completed", (i.e. there was a 
non-interim response given to the
client) although the request body would not have been sent. 


>
>> I think with the way security and encrytion are moving, we are moving 
>> more towards negotiation of credentials
>> on a per-connection basis.  Reuse of any crypto derived information is 
>> normally seen as a vulnerability, so being
>> able to reuse nonces etc is arguably a security hole.  I see pressure to 
>> move towards a nonce per connection, or in
>> other words session-based credentials like NTLM.  Therefore I see this 
>> "problem" growing rather than shrinking.
>
> To use connection oriented schemes effectively HTTP needs a bit or
> redesign. The protocol is not designed for connection oriented
> authentication, only message oriented authentication.

understood.  It has been hacked a bit by MS to allow for NTLM, and all 
windows browsers
now support NTLM (because it's commercial suicide not to).  So it's 
actually well supported out there.


>
>
>> Not just this case "where the body is unneededly transmitted before 
>> closing the connection" but in
>> all cases, whether the connection is closed or not, and it is in fact 
>> the intention of this spec that the
>> connection remain open.
>
> For the connection to remain open without redefining the HTTP message
> format or use chunked the request body MUST be transmitted and
> discarded.

I guess more changes to the spec would need to be made to allow for the 
request to be deemed complete
without having to send the request body.

So, yes, it then is a change to the semantics of client requests.

>
>>> It does not at all
>>> solve the problems of NTLM or other connection oriented authentication
>>> methods.
>>    
>> yes it does.
>
> Only if its short-circuiting the request skipping the request-body where
> HTTP message format requires one toe be sent, in which case the
> objections in the first message again applies and I did not misread what
> you wrote..
>
>> I think there's still some confusion about what I'm proposing.  
>> Hopefully that's cleared up with the
>> 3 cases I outlined above.
>
> I think I understood what you are proposing the first time, even if it's
> not what you have written.
>
> You propose changing the basic HTTP message format telling the client
> that if it received a 1xx Defer then it should not only wait
> indefinitely before sending the request body but also skip sending the
> request body and still keep the connection open if it receives a final
> response (i.e. auth challenge) before the body has been sent.
>
> Imho to do this you MUST use chunked encoding, and have the client
> terminate the request with a end chunk. Redefining the HTTP message
> format to allow skipping transmission of a Content-Length indicated
> request body is not possible within HTTP/1.1 as it requires the basic
> message format to be redefined to allow the body to be omitted.

Thanks for your patience with me!

OK, so you have what I am proposing there. 

It looks like I omitted some things from the spec.  But, since the spec 
would require changes
to HTTP, and clients and servers and proxies, then is it too much of a 
divergence to allow for
the spec to also then cover a relaxation in strict cases (clearly 
indicated) of the definition of completeness
of a request?

I foresee problems if the client arbitrarily terminates the request body 
by sending a final chunk.  The
proxy then has something that it no longer knows is complete or not.  It 
may have already sent
most of this to the server.  Who knows what the server will have done 
with it.

If the client supports the proposed spec though, we can avoid this 
altogether.

if the client doesn't support this proposal, we are where we are now.

If we are going to change the protocol, and change clients, then we may 
as well change them to provide
the best solution, not one that has known problems, simply to fall 
within the current letter of the current
HTTP spec which we would be changing anyway.  There should be no 
ambiguities where a client supporting this
encounters proxies and servers that dont either, so in all cases, the 
step to take should be clear.

So, I guess I should go away and make a second draft of the proposal 
which covers these aspects.

Anyone know of a decent I-D formatting client?  I may have to write 
one.  Took longer to manually format
pages etc than to write the content (maybe that shows...).

Adrien


>
> Regards
> Henrik

Received on Wednesday, 14 February 2007 03:10:06 UTC