Re: estimated Content-Length with chunked encoding

the use case relating to scanning at a proxy is actually very widespread.

For instance AV scanners that do heuristic scanning can't function 
without the entire resource, therefore at a gateway, the entire download 
must be spooled before it's safe to send any of it.  This means the user 
sits waiting until the proxy has downloaded the entire file and scanned 
it before they see ANY progress.

This affects millions of people, it's a big problem, and historically a 
big support issue for us.  Workarounds are almost worse than the problem 
- sending part of the file to keep the human attached to the browser 
happy that something is happening.  Trading security against convenience 
ad costing complexity and performance.

Another area we see problems in is in requesting resources which take a 
very long time to generate.  E.g. logging into a webmail server with a 
huge index and number of folders on the back end, it can take several 
minutes before a single byte is sent to the UA.  Humans don't wait that 
long.  They hit the refresh button several times before that.



Brian Smith wrote:
> Adrien de Croy wrote:
>   
>> William A. Rowe, Jr. wrote:
>>     
>>> Adrien de Croy wrote:
>>>       
>> could be interesting.  My original use case was to prevent user
>> timeouts due to lack of perceived progress, so all the 1xx responses
>> would be sent prior to any data (chunked or otherwise)... but I can
>> see some good potential for being able to write to the UI of a UA
>> during chunked data download as well - like a multiplexed data stream.
>>     
>
> By far the most common case, in my experience, is when the server is 
> dynamically compressing the response (e.g. mod_deflate). In that case, the 
> server often knows exactly the unencoded length but doesn't know the encoded 
> length. If the server could send the unencoded length in the header then the 
> client could provide a perfectly accurate download progress bar as long as it 
> decodes the content-encoding as it downloads. I would be very interested in 
> such an extension and I think UA implementers would be quick to implement it.
>
> I'm not sure that UA builders are going to care so much about the other 
> proposed uses. There are already workarounds for those use cases (using AJAX 
> and with the progress information encoded in custom message formats). In order 
> to be useful for AJAX applications, the information would have to be exposed 
> in the XHR object. Further, it is really confusing or the end user to see an 
> estimate of progress that is inaccurate, so I doubt UAs are going to be eager 
> to display anything that is just an estimate. It seems like a lot of 
> specification work, followed by a lot of implementation work, just to show the 
> user information that is often likely to be inaccurate and misleading.
>
>   
>> Although I thought chunk extension headers could only occur on the
>> final chunk?  Or are they different to attribute extensions... bit
>> rusty on those...
>>     
>
> I think you are thinking about trailers. Chunk extensions can occur on any 
> chunk, AFAICT, according to the specification. However, I wouldn't be 
> surprised to see real world interoperability problems with them.
>
> Regards,
> Brian
>   

-- 
Adrien de Croy - WinGate Proxy Server - http://www.wingate.com

Received on Tuesday, 21 October 2008 09:37:01 UTC