Re: http progress notification

On 25/05/2010 8:28 p.m., Henrik Nordström wrote:
> tis 2010-05-25 klockan 16:15 +1200 skrev Adrien de Croy:
>    
>> We've made some more great progress with testing on this, which has lead
>> to a fairly major change in the structure of information passed back by
>> servers.
>>      
> Looks good to me. Some comments
>
>
> Maybe add an optional Progress request header, allowing client to
> indicate for example how frequent it desires to get progress updates.
> But probably overkill as the clients needing this most also is low
> bandwidth clients and adding a header like this for those is
> counterproductive..
>
>    

Yes, the original I-D proposed this, but in the end I figured the most 
common use case would be where the proxy was well connected to the 
client, and the complexity it would add in the server was hardly worth it.

There are also applications for a server to use this directly to a 
client (e.g. when there's a lot of processing to do before sending a 
response, e.g. webmail).

> I also thought about perhaps an explicit note to not respond with 102 if
> the request was HTTP/1.0. But this should not be needed today. As far as
> I know all HTTP/1.0 proxies deployed today support the Connection
> header.
>    

we would only use this if we received a Progress token in the Connection 
header.  Do you mean some HTTP/1.0 proxies may pass the Connection 
header on verbatim so pass it on without understanding it?

I think even HTTP/1.0 required intermediaries and clients to support 
multiple 1xx responses, so this would probably even just work.

> "OK" is maybe not the most appropriate status code description, and
> confusingly similar to 200.. "Progress" or "Info" perhaps?
>    

sure

> Something which may require some additional thought is how to handle
> chains of proxies using Progress. For example Client<->  Transforming
> proxy<->  AV Proxy<->  Server. How should an intermediary forward
> progress information when it is itself also buffering and processing the
> response?
>    

Since the interim stage only occurs prior to the actual response (e.g. 
200 ok), it's not possible to be buffering an entity at the time you are 
receiving 1xx messages from upstream.

Therefore the only agent generating 1xx updates would be the agent 
currently doing the spooling.  This means filters inside a gateway can 
individually own the task of generating these.

Any downstream agent seeing 1xx messages going past doesn't really have 
anything meaningful to add, since they are just waiting for upstream.


> And another use case to cover for progress notifications is when
> intermediaries buffer the request before forwarding causing delays in
> request forwarding which in effect looks the same as delays in request.
>
>    

yes, this is an interesting use case, and one I think will also benefit 
greatly.

Uploading a large file where you scan at a gateway typically has 
problems where the file is spooled to the gateway extremely quickly, 
then scanned and slowly then uploaded to the target server before the 
server responds and that response is relayed back to the client.

This means the user experience is a very quick upload followed by a very 
long delay before a result.  Highly confusing for users.

Progress notifications in this scenario would therefore be also very 
useful so that the client could see the progress of the scan and upload 
to the server.

Actually the upload scenario lends itself to one more state: 
CONNECTING.  This would allow the proxy to buffer the entire request and 
scan it before even connecting to the origin server.

The best thing about interim responses (compared to drip-feeding part of 
the entity - the most common solution to the problem) is that the proxy 
can respond with say a 403 if it finds a virus, and the client hasn't 
received any (possibly compromised) entity body, yet has been able to 
display progress as if it were directly connected.


One issue we are seeing.   Browsers don't know how to treat the response 
until they sees Content-Type.  This means they don't know if it's a file 
download (display save as dialog, or similar) vs just some part of a 
page or other resource to display in-line.  Chrome for instance will 
abort non-download requests when you navigate to another page, whereas 
downloads continue.

So we've discussed here possibly sending the Content-Type (and possibly 
other headers) through to the client as well, so it can know whether 
it's downloading something or not, and behave accordingly.   I can't yet 
think of any requirement for other headers but we could perhaps define a 
way to transport the headers back without making them actual headers of 
the 1xx response, e.g. prefix header names with X-Orig- or encapsulate 
them somehow into the Progress response header.  If it were just 
Content-Type, we'd just put that through as type="..."

Have to also be careful about escaping and charsets for quoted text.

Regards

Adrien

> Regards
> Henrik
>
>
>    

-- 
Adrien de Croy - WinGate Proxy Server - http://www.wingate.com

Received on Tuesday, 25 May 2010 10:38:48 UTC