Re: Resend: HTTP Performance extension for NAV(network-based anti-virus) systems

Hi,

Thanks for the comments. Please see inline.


At 02:57 AM 8/7/2007, Adrien de Croy wrote:


>Hi
>
>The underlying issue IMO is that of the human user knowing what's 
>going on.  If you have some sort of progress notification, that 
>allows the progress bar on the bottom of the browser window to show 
>movement, and the status area to show activity, then users won't 
>hang up waiting.
>Download managers could show this information as well.

Agree. But I would like to clarify that solutions to "Progress 
notifications" or "clients inactivity timeout" is not the focus of 
this draft. This was only an offshoot of the approach used to solve 
the "performance issue". I will remove progress notification 
statements from the draft, if necessary.

>The actual transmission of the file from the proxy to the client is 
>then secondary.  It will happen at LAN speed once the file is scanned.

This is the main focus of this draft. Though it is perceived that 
downloading file at LAN speed is faster, some practical observations 
(under typical LAN congestion level & server load/processing 
capacity) indicated nearly 7.3% improvement in HTTP response time, if 
responses are sent to client while scanning is still in progress.

Consider this scenario:

HTTP client --LAN-- Edge -----1Mbps IPLC-----Edge--LAN--Edge ------ Internet
                |              from ISP              |
             BrachOffice                          MainOffice
             webServer                            Webserevr
<---branch office in -->                     <-----Main ---->
     INDIA(100 Mbps LAN speed)                  office in US
                                                (100 Mbps LAN)

In this scenario, it is observerd that
     - A 12.4 MB file download from an internet webserver to "Branch 
WebServer" took 154 sec.
     - clamd took 11 sec the 12.4 MB file.
     - Client took 13 sec to download the same 12.4 MB file.

These figures indicate that
     - HTTP response time without a NAV proxy: 154 sec
     - HTTP response time with NAV proxy: 154 + 11(clamd scan time) + 
13 (proxy->client transfer time) = 178 sec

The focus of this draft is to lessen the 13 sec from this HTTP 
response time, by sending the HTTP response from Internal web proxy 
to the client even before scanning is complete.

In this scenario 13 sec amounts to nearly 7.3 %.

Consider the following scenarios:
     - Branch office and main office are connected though IPLC. NAV 
scanning gets applied at a proxy in main office. Here the time it 
takes for the proxy to send HTTP response to a client in brach office 
will be more than that seen in the above example.
     - Branch office and main office are connected via a L3 VPN. NAV 
scanning gets applied at a proxy in main office. Again, the time it 
takes for the proxy to send HTTP response to a client in brach office 
will be more than that seen in the above examples.

This percentage may increase/decrease depending on NAV proxy->Client 
network speed, congestion level, NAV proxy load & capacity.

Please let me know you thoughts.


Thanks,
Babu


>Since responses to date have been interim, the result of the scan 
>(e.g. if failure) can simply be returned as the response to the request.
>
>Your proposed method seems to assume that transfer of the resource 
>is necessary for there to be progress to show.  Having a separate 
>specific progress mechanism however does this, and furthermore is 
>applicable to more cases than just gateway scanning, in fact is 
>useful any time a request will take a while to process, and can be 
>generated by gateways and/or servers.
>
>Furthermore the Progress mechanism is transparent to proxies that 
>don't understand it, and is much less complicated to implement than 
>a multi-part post-abort mechanism, so would find much easier deployment.
>It's arguably safer as well, since a problem in post-aborting a 
>request could still leave compromised data on the client system, 
>puts the client to more work as well (esp considering client-side 
>processing such as AV, caching etc).
>
>Regards
>
>Adrien
>
>
>Babu.N wrote:
>>Hi,
>>
>>Thanks for the comments.
>>
>>Please see inline.
>>
>>At 06:41 AM 8/3/2007, Adrien de Croy wrote:
>>
>>
>>>Hi
>>>
>>>A few months back I submitted an I-D which contained proposals for 
>>>progress notifications.
>>>
>>>the reason I raise that, is because it considers the same 
>>>scenarios as your draft, i.e. it was specifically conceived to 
>>>deal with issues around gateway scanning.
>>>
>>>There is a problem with the approach taken by your I-D.
>>>
>>>It is concerned around the issue of event sending any of the 
>>>resource to the client prior to it having been verified as safe.
>>>
>>>Trying to get the client to ignore the file after it has 
>>>downloaded a whole heap of it isn't safe.  The file may have in 
>>>fact been padded out to avert such techniques, so an entire 
>>>workable image may have been already received by the client, which 
>>>would run if executed.
>>
>>This is the reason I have the following statements in the draft:
>>
>>"
>>    For some reason, if a client doesn't receive second and
>>third (when    applicable) sub-parts, it MUST never store/process the
>>already    received data.
>>
>>"
>>
>>"
>>    Some programs like browsers' "Download
>>Manager" start storing the    data to disk even before the complete 
>>response is received.
>>Such    applications should ensure that they delete any partial data
>>in the    event of "Proxy response IGNORE" or when
>>second/third body (when    applicable) parts are not received.
>>"
>>
>>So implementations confirming to this strictly should never 
>>run/process a response till the response is completely received. So 
>>they should not have an issue ? Please suggest.
>>
>>
>>Thanks,
>>Babu
>>
>>
>>>There's not really any amount of an unscanned resource that can be 
>>>guaranteed to be safe to send (well maybe just a few bytes).
>>>
>>>This is why we're trying to deprecate "drip-feeding" in WinGate, 
>>>since it's fundamentally a flawed concept, albeit necessary to 
>>>appease the users of browsers.
>>>
>>>I just submitted a revised draft which I'll also post here, which 
>>>just focuses on using interim response messages to report progress.
>>>
>>>Cheers
>>>
>>>Adrien
>>>
>>>
>>>babun@intoto.com wrote:
>>>>Hi,
>>>>
>>>>I have submitted a draft on "HTTP Performance extension for NAV
>>>>(network-based anti-virus) systems":
>>>>http://www.ietf.org/internet-drafts/draft-babu-navmime-00.txt
>>>>
>>>>This draft attempts to also solve issues in reporting download progress
>>>>and avoiding client connection timeout in presence of network-based
>>>>anti-virus/anti-spam proxies.
>>>>
>>>>Like to hear feedback on this.
>>>>
>>>>
>>>>Thanks,
>>>>Babu
>>>>
>>>>PS: Appologies for resending this. Forgot to indicate the subject in my
>>>>last email, hence re-sending.
>>>>
>>>>
>>>>
>>>>********************************************************************************
>>>>This email message (including any attachments) is for the sole 
>>>>use of the intended recipient(s) and may contain confidential, 
>>>>proprietary and privileged information. Any unauthorized review, 
>>>>use, disclosure or distribution is prohibited. If you are not the 
>>>>intended recipient, please immediately notify the sender by reply 
>>>>email and destroy all copies of the original message. Thank you.
>>>>
>>>>Intoto Inc.
>>>>
>>>>
>>>
>>>--
>>>Adrien de Croy - WinGate Proxy Server - http://www.wingate.com 
>>><http://www.wingate.com/>
>
>--
>Adrien de Croy - WinGate Proxy Server - http://www.wingate.com


********************************************************************************
This email message (including any attachments) is for the sole use of the intended recipient(s) 
and may contain confidential, proprietary and privileged information. Any unauthorized review, 
use, disclosure or distribution is prohibited. If you are not the intended recipient, 
please immediately notify the sender by reply email and destroy all copies of the original message. 
Thank you.
 
Intoto Inc.