W3C home > Mailing lists > Public > ietf-http-wg@w3.org > January to March 2012

Re: #311 Add limitations to Range to reduce its use as a denial-of-service tool

From: Mark Nottingham <mnot@mnot.net>
Date: Tue, 7 Feb 2012 13:53:05 +1100
Cc: ietf-http-wg@w3.org
Message-Id: <DBB37FBF-F6FD-4950-BBC8-800482C779E6@mnot.net>
To: Adrien de Croy <adrien@qbik.com>
The Introduction already says:

> The protocol for range requests
>    is an OPTIONAL feature of HTTP, designed so resources or recipients
>    that do not implement this feature can respond as if it is a normal
>    GET request without impacting interoperability.


Cheers,


On 05/01/2012, at 1:46 PM, Adrien de Croy wrote:

> 
> while we're on the topic of range requests, there are an increasing number of agents that fail when range requests are responded to with a 200 and full content.
> 
> Things like
> 
> * MS Windows update
> * iTunes
> * AppleTV client
> * various other updaters
> * anything relying on BITS in Windows.
> 
> etc.
> 
> This is a conflict with any gateway AV scanning which requires the entire entity in order to scan it and which therefore removes Range headers from requests.
> 
> Could/should we add language to send a stronger message to agent authors to deter them from such behaviour?  It makes little or no sense to post an error about a misconfigured gateway simply because it downgrades all range requests to full requests.  The entire entity is still available to be sent back to the client, it just refuses to play if it can't get its way.
> 
> it should be clear that it's entirely valid for a server to respond to a Range request with 200 and full content, and entirely valid for a server to supply a single range response to even a multiple range request, and that range may or may not even cover all the ranges requested.  It should be entirely up to any client which wishes to use ranges to sort out what bits it gets out of what it asked for and make subsequent requests accordingly.
> 
> Adrien
> 
> 
> On 5/01/2012 2:24 p.m., Amos Jeffries wrote:
>> On Wed, 4 Jan 2012 19:18:59 -0500 (EST), Yves Lafon wrote:
>>> On Thu, 5 Jan 2012, Amos Jeffries wrote:
>>> 
>>>> On Wed, 4 Jan 2012 09:00:38 -0500 (EST), Yves Lafon wrote:
>>>>> Here is a proposal to clarify a few things to help mitigating the
>>>>> overlapping range DoS issue:
>>>>> 1/ split section 4 in Combining Range in Caches (the actual content)
>>>>>   and add a specific section about combining ranges in responses.
>>>>>   (with a link to security considerations, and link back from security
>>>>>   section to this newly created section.
>>>>> 2/ Responses to single and multiple range requests / Combining Ranges
>>>>> in a response.
>>>>> Add:
>>>>> <<
>>>>>   Servers have liability to combine requested ranges when those ranges
>>>>>   are ovelapping.
>>>>>>> 
>>>> 
>>>> Is the phrasing "have liability" defined somewhere? I may have missed it, but it still took me a few minutes to realise what this sentence meant despite being part of the discussions earlier around Range.
>>> 
>>> Hum, I will rewrite that, but you got the spirit of it.
>>> 
>> 
>> Yes. Cheers. :)
>> 
>>>>> [Move this part of 5.2 here]
>>>>> <<
>>>>>   When an HTTP message includes the content of a single range (for
>>>>>   example, a response to a request for a single range, or to a request
>>>>>   for a set of ranges that overlap without any holes), this content is
>>>>>   transmitted with a Content-Range header field, and a Content-Length
>>>>>   header field showing the number of bytes actually transferred.  For
>>>>>   example,
>>>>> 
>>>>>     HTTP/1.1 206 Partial Content
>>>>>     Date: Wed, 15 Nov 1995 06:25:24 GMT
>>>>>     Last-Modified: Wed, 15 Nov 1995 04:58:08 GMT
>>>>>     Content-Range: bytes 21010-47021/47022
>>>>>     Content-Length: 26012
>>>>>     Content-Type: image/gif
>>>>> 
>>>>>   When an HTTP message includes the content of multiple ranges (for
>>>>>   example, a response to a request for multiple non-overlapping
>>>>>   ranges), these are transmitted as a multipart message.  The multipart
>>>>>   media type used for this purpose is "multipart/byteranges" as defined
>>>>>   in Appendix A.
>>>>> 
>>>>>   A response to a request for a single range MUST NOT be sent using the
>>>>>   multipart/byteranges media type.  A response to a request for
>>>>>   multiple ranges, whose result is a single range, MAY be sent as a
>>>>>   multipart/byteranges media type with one part.  A client that cannot
>>>>>   decode a multipart/byteranges message MUST NOT ask for multiple
>>>>>   ranges in a single request.
>>>>> 
>>>>>   When a client requests multiple ranges in one request, the server
>>>>>   SHOULD return them in the order that they appeared in the request.
>>>>>>>> 
>>>> 
>>>> This last part seems to contradict my understanding of the last discussion consensus. That ranges could be combined to reduce DoS footprint.
>>>> 
>>>> Did I get that completely out of place? or is this use of SHOULD a downgrade intended to permit it?
>>> 
>>> I don't see a contradiction there, it just says that the ranges should be
>>> sent in order, however when ranges are combined, ordering can become
>>> invalid (if the combined ranges were not directly following each
>> 
>> invalid? I don't see that in the new texts. single-part "multipart" is MAY, which should at most solve any validity issue on merging.
>> 
>>> other), so yes, it's the use of a SHOULD isntead of a MUST that allows
>>> that, while giving general guidelines for the general "normal" case.
>>> 
>> 
>> Thanks. That answers the initial Q.
>> 
>> AYJ
>> 
> 
> -- 
> Adrien de Croy - WinGate Proxy Server - http://www.wingate.com
> WinGate 7 is released! - http://www.wingate.com/getlatest/
> 
> 

--
Mark Nottingham   http://www.mnot.net/
Received on Tuesday, 7 February 2012 02:59:16 GMT

This archive was generated by hypermail 2.2.0+W3C-0.50 : Friday, 27 April 2012 06:51:55 GMT