Re: Using HTTP Trailers [was: Content-Integrity header]

On Wed, Jul 11, 2012 at 2:30 PM, Zhong Yu <zhong.j.yu@gmail.com> wrote:
> If the value of a trailer is a function of the message body, and the
> function is app specific, it's a little hard for filters (who
> intercept and transform responses) to correctly transform the value of
> the trailer.

The objective of using a MAC is to detect such modification so that
the messages can be rejected.

It isn't something I would see a lot of use for in Web browsing but
there is a huge need for the capability in Web Services where people
seem to keep inventing additional packaging layers just to fit the
security data in.

For example, take a look at JSON signature where we are being asked to
BASE64 encode stuff that could and should be binary.


> On Wed, Jul 11, 2012 at 3:01 AM, Wenbo Zhu <wenboz@google.com> wrote:
>>
>>
>> On Tue, Jul 10, 2012 at 7:50 PM, James M Snell <jasnell@gmail.com> wrote:
>>>
>>> Yeah, this has been my experience as well. Unfortunately, I'm not sure
>>> if this is something that can be effectively fixed. Off the top of my
>>> head, I don't know of a single server or client side http stack that
>>> is capable of easily setting Trailers in the stream. I just checked
>>> and support for trailers is still missing in the Servlet API, for
>>> instance.
>>
>> There is no new API required, necessarily, to support trailers:
>> 1. app code needs set the header Trailer: <header_name> before the response
>> is committed;
>> 2. container will ignore the header  <header_name> when the response is
>> committed;
>> 3. container will output the trailer when the response is finished.
>>
>> The request header "TE: trailers" may or may not be respected.
>>
>>
>>>
>>> I know this kind of thing is generally frowned upon, but one possible
>>> approach is to make use of a special content-type... for instance...
>>>
>>>   HTTP/1.1 200 OK
>>>   Trailer: Content-Integrity
>>>   Content-Type: application/http-trailers; type="text/plain"
>>>
>>>   foobar
>>>
>>>   Content-Integrity:
>>> sha-256:aec070645fe53ee3b3763059376134f058cc337247c978add178b6ccdfb0019f
>>>
>>> This is just a strawman, but essentially, it would append a checksum
>>> to the end of the content stream. It's not a trailer in the
>>> traditional sense, it's actually encoded as part of the payload. In
>>> lieu of proper Trailer support this is the only other way I can see it
>>> working effectively in HTTP/1.1.
>>>
>>> For HTTP/2.0, I suppose it would be possible for us to take a more
>>> appropriate approach either with trailers or by building the integrity
>>> mechanism into the framing somehow.
>>>
>>> (Warning: thinking out loud *before* having consumed any beer... the
>>> ideas expressed above may not make any sense. May need to apply
>>> alcohol and try again)
>>>
>>> - James
>>>
>>> On Tue, Jul 10, 2012 at 4:59 PM, Mark Nottingham <mnot@mnot.net> wrote:
>>> > chunk-extensions aren't widely supported in APIs, and usually dropped
>>> > hop-by-hop (then again, trailers can be too).
>>> >
>>> > Some discussion of trailer support in Apache:
>>> >
>>> > http://apache-http-server.18135.n6.nabble.com/HTTP-trailers-td4796242.html
>>> >
>>> > … and using Trailers to surface debug information in Firebug:
>>> >   https://groups.google.com/forum/?fromgroups#!topic/firebug/v5ldjoVThH8
>>> >
>>> > (yes, this is a bit of a project for me)
>>> >
>>> > Finally, I know some server-side implementers would would LOVE to be
>>> > able to set things like ETag and Last-Modified in trailers, and have them
>>> > used by caches...
>>> >
>>> > Cheers,
>>> >
>>> >
>>> > On 11/07/2012, at 4:33 AM, Ludin, Stephen wrote:
>>> >
>>> >> I really like the idea of placing the Digest in the chunk trailers.
>>> >> Being
>>> >> able to calculate these digests on the fly and not buffer the entire
>>> >> message is critical in my opinion.
>>> >>
>>> >> Another concept that I have been playing with is providing digests on
>>> >> individual chunks using chunk-extension.  The rational for this is for
>>> >> very large objects.  With per-chunk digests the client would have the
>>> >> ability to re-request a specific corrupted section of an object using a
>>> >> range request rather than the entire object.  This can have enormous
>>> >> perceived performance and reliability benefits for consumers of things
>>> >> such as software download and large media files.
>>> >>
>>> >> I was working on a draft to propose this, but I didn't feel it was well
>>> >> baked enough to share.  If there is interest in this type of
>>> >> functionality
>>> >> I will polish it up and post it.
>>> >>
>>> >> One issue to point is is that for these types of "frame" based
>>> >> integrity
>>> >> checks I generally feel like we are reinventing the content integrity
>>> >> portion of SSL/TLS.  Though I see the value in begin able to do this
>>> >> apart
>>> >> from SSL it forces the question at what point do you just switch over
>>> >> to
>>> >> SSL to get the desired functionality?
>>> >>
>>> >> -stephen
>>> >>
>>> >>
>>> >>
>>> >> On 7/9/12 4:00 PM, "Amos Jeffries" <squid3@treenet.co.nz> wrote:
>>> >>
>>> >>> On 10.07.2012 07:08, HAYASHI, Tatsuya wrote:
>>> >>>> +1
>>> >>>>
>>> >>>> I know that this is demanded.
>>> >>>> When I discussion about http-authentication and phishing,
>>> >>>> it is requested by many people.
>>> >>>> It is a difficult problem.
>>> >>>> ex) proxy...
>>> >>>>
>>> >>>> I think that it is good to do this discussion now.
>>> >>>>
>>> >>>> ---
>>> >>>> Tatsuya
>>> >>>>
>>> >>>> On Sat, Jul 7, 2012 at 8:23 AM, James M Snell wrote:
>>> >>>>> In general, I'm +1 on the general idea albeit with a few caveats...
>>> >>>>>
>>> >>>>> 1. To minimize complexity, only a single Content-Integrity header
>>> >>>>> should be used. I don't want, as Roy points out, to have to iterated
>>> >>>>> through a bunch of unsupported header values looking for the one I
>>> >>>>> want. Just as it makes very little sense for an implementor to
>>> >>>>> provide
>>> >>>>> multiple Last-Modified, Etag and Content-Type headers in a single
>>> >>>>> message; there should be only a single Content-Integrity statement
>>> >>>>> and
>>> >>>>> I either understand it or I don't.
>>> >>>
>>> >>> Either the client advertises what it supports (opening itself to
>>> >>> middleware erasing options they can't modify). Or the server uses
>>> >>> multiple algorithms in hopes that the middleware cannot violate them
>>> >>> all.
>>> >>> It makes perfect sense to have several levels of integrity check. MD5,
>>> >>> SHA1, AES in one response and allow the client to validate the
>>> >>> strongest
>>> >>> it can handle.
>>> >>>
>>> >>> There is also an arguable case for middleware wanting to add its own
>>> >>> hash to inform the client essentially "this is what I got given". So
>>> >>> the
>>> >>> point of manipulation can be back-traced when the more secure
>>> >>> end-to-end
>>> >>> checks fail.
>>> >>>
>>> >>> If you want end-to-end integrity, don't stop at half measures.
>>> >>> Particularly at half measures which can be corrupted.
>>> >>>
>>> >>>
>>> >>>>>
>>> >>>>> 2. The performance impact of calculating the digest needs to be
>>> >>>>> carefully considered. I'd rather not be required to buffer a full
>>> >>>>> representation in memory all the time just to calculate a header
>>> >>>>> value. I know it's largely unavoidable, but perhaps there's some
>>> >>>>> currently elusive solution that can be considered. For instance..
>>> >>>>> allowing Content-Integrity to appear as a trailer at the end of a
>>> >>>>> chunked response.
>>> >>>
>>> >>> As has been said Trailers happen here.
>>> >>>
>>> >>>>>
>>> >>>>> 3. Something needs to be said about what happens if the
>>> >>>>> Content-Integrity check fails. For instance, if a request containing
>>> >>>>> Content-Integrity is sent to the server and the server detects that
>>> >>>>> the signature is invalid, what should happen? what must happen?
>>> >>>>> Likewise, how are intermediaries expected to treat the
>>> >>>>> Content-Integrity header given that any intermediary is able to
>>> >>>>> modify
>>> >>>>> the payload at any time?
>>> >>>
>>> >>> This is going to be most useful on request/responses sent with
>>> >>> "no-transform" of course.
>>> >>> If the integrity was only a MD5 or SHA1 hash which middleware can edit
>>> >>> easily there is no end-to-end integrity, just hop-by-hop integrity.
>>> >>>
>>> >>>
>>> >>> Also, there has to be a mutual secret between origin server and
>>> >>> client.
>>> >>> Without that, when integrity is compromised the transforming hop will
>>> >>> simply erase or replace the Content-Integrity header value. A secret
>>> >>> key
>>> >>> unknown to that middleware is required to make the integrity hash
>>> >>> break
>>> >>> when it tries this.
>>> >>>
>>> >>> AYJ
>>> >>>
>>> >>
>>> >>
>>> >
>>> > --
>>> > Mark Nottingham
>>> > http://www.mnot.net/
>>> >
>>> >
>>> >
>>> >
>>> >
>>>
>>
>



-- 
Website: http://hallambaker.com/

Received on Wednesday, 11 July 2012 22:38:43 UTC