W3C home > Mailing lists > Public > ietf-http-wg@w3.org > July to September 2012

Re: Using HTTP Trailers [was: Content-Integrity header]

From: James M Snell <jasnell@gmail.com>
Date: Tue, 10 Jul 2012 20:52:53 -0700
Message-ID: <CABP7RbeGKDJ5yV-P4yxG954yE2-X7=EOJ9QYKvMEjpss0q4_3A@mail.gmail.com>
To: Mark Nottingham <mnot@mnot.net>
Cc: "Ludin, Stephen" <sludin@akamai.com>, Amos Jeffries <squid3@treenet.co.nz>, "ietf-http-wg@w3.org" <ietf-http-wg@w3.org>
On Tue, Jul 10, 2012 at 7:50 PM, James M Snell <jasnell@gmail.com> wrote:
> [snip]
> I know this kind of thing is generally frowned upon, but one possible
> approach is to make use of a special content-type... for instance...
>
>   HTTP/1.1 200 OK
>   Trailer: Content-Integrity
>   Content-Type: application/http-trailers; type="text/plain"
>
>   foobar
>
>   Content-Integrity:
> sha-256:aec070645fe53ee3b3763059376134f058cc337247c978add178b6ccdfb0019f
>

Oh, forgive me, I had a beer and realized I made an error in the
sample, the content-type really should be...

  application/ugly-hack-to-work-around-broken-implementations+http

Without either a hack of this nature or a new Transfer-Encoding that
we could somehow convince folks to support, there's really no other
option but to punt and make trailers required in 2.0.

- James

> This is just a strawman, but essentially, it would append a checksum
> to the end of the content stream. It's not a trailer in the
> traditional sense, it's actually encoded as part of the payload. In
> lieu of proper Trailer support this is the only other way I can see it
> working effectively in HTTP/1.1.
>
> For HTTP/2.0, I suppose it would be possible for us to take a more
> appropriate approach either with trailers or by building the integrity
> mechanism into the framing somehow.
>
> (Warning: thinking out loud *before* having consumed any beer... the
> ideas expressed above may not make any sense. May need to apply
> alcohol and try again)
>
> - James
>
> On Tue, Jul 10, 2012 at 4:59 PM, Mark Nottingham <mnot@mnot.net> wrote:
>> chunk-extensions aren't widely supported in APIs, and usually dropped hop-by-hop (then again, trailers can be too).
>>
>> Some discussion of trailer support in Apache:
>>   http://apache-http-server.18135.n6.nabble.com/HTTP-trailers-td4796242.html
>>
>> … and using Trailers to surface debug information in Firebug:
>>   https://groups.google.com/forum/?fromgroups#!topic/firebug/v5ldjoVThH8
>>
>> (yes, this is a bit of a project for me)
>>
>> Finally, I know some server-side implementers would would LOVE to be able to set things like ETag and Last-Modified in trailers, and have them used by caches...
>>
>> Cheers,
>>
>>
>> On 11/07/2012, at 4:33 AM, Ludin, Stephen wrote:
>>
>>> I really like the idea of placing the Digest in the chunk trailers.  Being
>>> able to calculate these digests on the fly and not buffer the entire
>>> message is critical in my opinion.
>>>
>>> Another concept that I have been playing with is providing digests on
>>> individual chunks using chunk-extension.  The rational for this is for
>>> very large objects.  With per-chunk digests the client would have the
>>> ability to re-request a specific corrupted section of an object using a
>>> range request rather than the entire object.  This can have enormous
>>> perceived performance and reliability benefits for consumers of things
>>> such as software download and large media files.
>>>
>>> I was working on a draft to propose this, but I didn't feel it was well
>>> baked enough to share.  If there is interest in this type of functionality
>>> I will polish it up and post it.
>>>
>>> One issue to point is is that for these types of "frame" based integrity
>>> checks I generally feel like we are reinventing the content integrity
>>> portion of SSL/TLS.  Though I see the value in begin able to do this apart
>>> from SSL it forces the question at what point do you just switch over to
>>> SSL to get the desired functionality?
>>>
>>> -stephen
>>>
>>>
>>>
>>> On 7/9/12 4:00 PM, "Amos Jeffries" <squid3@treenet.co.nz> wrote:
>>>
>>>> On 10.07.2012 07:08, HAYASHI, Tatsuya wrote:
>>>>> +1
>>>>>
>>>>> I know that this is demanded.
>>>>> When I discussion about http-authentication and phishing,
>>>>> it is requested by many people.
>>>>> It is a difficult problem.
>>>>> ex) proxy...
>>>>>
>>>>> I think that it is good to do this discussion now.
>>>>>
>>>>> ---
>>>>> Tatsuya
>>>>>
>>>>> On Sat, Jul 7, 2012 at 8:23 AM, James M Snell wrote:
>>>>>> In general, I'm +1 on the general idea albeit with a few caveats...
>>>>>>
>>>>>> 1. To minimize complexity, only a single Content-Integrity header
>>>>>> should be used. I don't want, as Roy points out, to have to iterated
>>>>>> through a bunch of unsupported header values looking for the one I
>>>>>> want. Just as it makes very little sense for an implementor to
>>>>>> provide
>>>>>> multiple Last-Modified, Etag and Content-Type headers in a single
>>>>>> message; there should be only a single Content-Integrity statement
>>>>>> and
>>>>>> I either understand it or I don't.
>>>>
>>>> Either the client advertises what it supports (opening itself to
>>>> middleware erasing options they can't modify). Or the server uses
>>>> multiple algorithms in hopes that the middleware cannot violate them
>>>> all.
>>>> It makes perfect sense to have several levels of integrity check. MD5,
>>>> SHA1, AES in one response and allow the client to validate the strongest
>>>> it can handle.
>>>>
>>>> There is also an arguable case for middleware wanting to add its own
>>>> hash to inform the client essentially "this is what I got given". So the
>>>> point of manipulation can be back-traced when the more secure end-to-end
>>>> checks fail.
>>>>
>>>> If you want end-to-end integrity, don't stop at half measures.
>>>> Particularly at half measures which can be corrupted.
>>>>
>>>>
>>>>>>
>>>>>> 2. The performance impact of calculating the digest needs to be
>>>>>> carefully considered. I'd rather not be required to buffer a full
>>>>>> representation in memory all the time just to calculate a header
>>>>>> value. I know it's largely unavoidable, but perhaps there's some
>>>>>> currently elusive solution that can be considered. For instance..
>>>>>> allowing Content-Integrity to appear as a trailer at the end of a
>>>>>> chunked response.
>>>>
>>>> As has been said Trailers happen here.
>>>>
>>>>>>
>>>>>> 3. Something needs to be said about what happens if the
>>>>>> Content-Integrity check fails. For instance, if a request containing
>>>>>> Content-Integrity is sent to the server and the server detects that
>>>>>> the signature is invalid, what should happen? what must happen?
>>>>>> Likewise, how are intermediaries expected to treat the
>>>>>> Content-Integrity header given that any intermediary is able to
>>>>>> modify
>>>>>> the payload at any time?
>>>>
>>>> This is going to be most useful on request/responses sent with
>>>> "no-transform" of course.
>>>> If the integrity was only a MD5 or SHA1 hash which middleware can edit
>>>> easily there is no end-to-end integrity, just hop-by-hop integrity.
>>>>
>>>>
>>>> Also, there has to be a mutual secret between origin server and client.
>>>> Without that, when integrity is compromised the transforming hop will
>>>> simply erase or replace the Content-Integrity header value. A secret key
>>>> unknown to that middleware is required to make the integrity hash break
>>>> when it tries this.
>>>>
>>>> AYJ
>>>>
>>>
>>>
>>
>> --
>> Mark Nottingham
>> http://www.mnot.net/
>>
>>
>>
>>
>>
Received on Wednesday, 11 July 2012 03:53:41 GMT

This archive was generated by hypermail 2.2.0+W3C-0.50 : Wednesday, 11 July 2012 03:53:49 GMT