Re: Using HTTP Trailers [was: Content-Integrity header]

If the value of a trailer is a function of the message body, and the
function is app specific, it's a little hard for filters (who
intercept and transform responses) to correctly transform the value of
the trailer.


On Wed, Jul 11, 2012 at 3:01 AM, Wenbo Zhu <wenboz@google.com> wrote:
>
>
> On Tue, Jul 10, 2012 at 7:50 PM, James M Snell <jasnell@gmail.com> wrote:
>>
>> Yeah, this has been my experience as well. Unfortunately, I'm not sure
>> if this is something that can be effectively fixed. Off the top of my
>> head, I don't know of a single server or client side http stack that
>> is capable of easily setting Trailers in the stream. I just checked
>> and support for trailers is still missing in the Servlet API, for
>> instance.
>
> There is no new API required, necessarily, to support trailers:
> 1. app code needs set the header Trailer: <header_name> before the response
> is committed;
> 2. container will ignore the header  <header_name> when the response is
> committed;
> 3. container will output the trailer when the response is finished.
>
> The request header "TE: trailers" may or may not be respected.
>
>
>>
>> I know this kind of thing is generally frowned upon, but one possible
>> approach is to make use of a special content-type... for instance...
>>
>>   HTTP/1.1 200 OK
>>   Trailer: Content-Integrity
>>   Content-Type: application/http-trailers; type="text/plain"
>>
>>   foobar
>>
>>   Content-Integrity:
>> sha-256:aec070645fe53ee3b3763059376134f058cc337247c978add178b6ccdfb0019f
>>
>> This is just a strawman, but essentially, it would append a checksum
>> to the end of the content stream. It's not a trailer in the
>> traditional sense, it's actually encoded as part of the payload. In
>> lieu of proper Trailer support this is the only other way I can see it
>> working effectively in HTTP/1.1.
>>
>> For HTTP/2.0, I suppose it would be possible for us to take a more
>> appropriate approach either with trailers or by building the integrity
>> mechanism into the framing somehow.
>>
>> (Warning: thinking out loud *before* having consumed any beer... the
>> ideas expressed above may not make any sense. May need to apply
>> alcohol and try again)
>>
>> - James
>>
>> On Tue, Jul 10, 2012 at 4:59 PM, Mark Nottingham <mnot@mnot.net> wrote:
>> > chunk-extensions aren't widely supported in APIs, and usually dropped
>> > hop-by-hop (then again, trailers can be too).
>> >
>> > Some discussion of trailer support in Apache:
>> >
>> > http://apache-http-server.18135.n6.nabble.com/HTTP-trailers-td4796242.html
>> >
>> > … and using Trailers to surface debug information in Firebug:
>> >   https://groups.google.com/forum/?fromgroups#!topic/firebug/v5ldjoVThH8
>> >
>> > (yes, this is a bit of a project for me)
>> >
>> > Finally, I know some server-side implementers would would LOVE to be
>> > able to set things like ETag and Last-Modified in trailers, and have them
>> > used by caches...
>> >
>> > Cheers,
>> >
>> >
>> > On 11/07/2012, at 4:33 AM, Ludin, Stephen wrote:
>> >
>> >> I really like the idea of placing the Digest in the chunk trailers.
>> >> Being
>> >> able to calculate these digests on the fly and not buffer the entire
>> >> message is critical in my opinion.
>> >>
>> >> Another concept that I have been playing with is providing digests on
>> >> individual chunks using chunk-extension.  The rational for this is for
>> >> very large objects.  With per-chunk digests the client would have the
>> >> ability to re-request a specific corrupted section of an object using a
>> >> range request rather than the entire object.  This can have enormous
>> >> perceived performance and reliability benefits for consumers of things
>> >> such as software download and large media files.
>> >>
>> >> I was working on a draft to propose this, but I didn't feel it was well
>> >> baked enough to share.  If there is interest in this type of
>> >> functionality
>> >> I will polish it up and post it.
>> >>
>> >> One issue to point is is that for these types of "frame" based
>> >> integrity
>> >> checks I generally feel like we are reinventing the content integrity
>> >> portion of SSL/TLS.  Though I see the value in begin able to do this
>> >> apart
>> >> from SSL it forces the question at what point do you just switch over
>> >> to
>> >> SSL to get the desired functionality?
>> >>
>> >> -stephen
>> >>
>> >>
>> >>
>> >> On 7/9/12 4:00 PM, "Amos Jeffries" <squid3@treenet.co.nz> wrote:
>> >>
>> >>> On 10.07.2012 07:08, HAYASHI, Tatsuya wrote:
>> >>>> +1
>> >>>>
>> >>>> I know that this is demanded.
>> >>>> When I discussion about http-authentication and phishing,
>> >>>> it is requested by many people.
>> >>>> It is a difficult problem.
>> >>>> ex) proxy...
>> >>>>
>> >>>> I think that it is good to do this discussion now.
>> >>>>
>> >>>> ---
>> >>>> Tatsuya
>> >>>>
>> >>>> On Sat, Jul 7, 2012 at 8:23 AM, James M Snell wrote:
>> >>>>> In general, I'm +1 on the general idea albeit with a few caveats...
>> >>>>>
>> >>>>> 1. To minimize complexity, only a single Content-Integrity header
>> >>>>> should be used. I don't want, as Roy points out, to have to iterated
>> >>>>> through a bunch of unsupported header values looking for the one I
>> >>>>> want. Just as it makes very little sense for an implementor to
>> >>>>> provide
>> >>>>> multiple Last-Modified, Etag and Content-Type headers in a single
>> >>>>> message; there should be only a single Content-Integrity statement
>> >>>>> and
>> >>>>> I either understand it or I don't.
>> >>>
>> >>> Either the client advertises what it supports (opening itself to
>> >>> middleware erasing options they can't modify). Or the server uses
>> >>> multiple algorithms in hopes that the middleware cannot violate them
>> >>> all.
>> >>> It makes perfect sense to have several levels of integrity check. MD5,
>> >>> SHA1, AES in one response and allow the client to validate the
>> >>> strongest
>> >>> it can handle.
>> >>>
>> >>> There is also an arguable case for middleware wanting to add its own
>> >>> hash to inform the client essentially "this is what I got given". So
>> >>> the
>> >>> point of manipulation can be back-traced when the more secure
>> >>> end-to-end
>> >>> checks fail.
>> >>>
>> >>> If you want end-to-end integrity, don't stop at half measures.
>> >>> Particularly at half measures which can be corrupted.
>> >>>
>> >>>
>> >>>>>
>> >>>>> 2. The performance impact of calculating the digest needs to be
>> >>>>> carefully considered. I'd rather not be required to buffer a full
>> >>>>> representation in memory all the time just to calculate a header
>> >>>>> value. I know it's largely unavoidable, but perhaps there's some
>> >>>>> currently elusive solution that can be considered. For instance..
>> >>>>> allowing Content-Integrity to appear as a trailer at the end of a
>> >>>>> chunked response.
>> >>>
>> >>> As has been said Trailers happen here.
>> >>>
>> >>>>>
>> >>>>> 3. Something needs to be said about what happens if the
>> >>>>> Content-Integrity check fails. For instance, if a request containing
>> >>>>> Content-Integrity is sent to the server and the server detects that
>> >>>>> the signature is invalid, what should happen? what must happen?
>> >>>>> Likewise, how are intermediaries expected to treat the
>> >>>>> Content-Integrity header given that any intermediary is able to
>> >>>>> modify
>> >>>>> the payload at any time?
>> >>>
>> >>> This is going to be most useful on request/responses sent with
>> >>> "no-transform" of course.
>> >>> If the integrity was only a MD5 or SHA1 hash which middleware can edit
>> >>> easily there is no end-to-end integrity, just hop-by-hop integrity.
>> >>>
>> >>>
>> >>> Also, there has to be a mutual secret between origin server and
>> >>> client.
>> >>> Without that, when integrity is compromised the transforming hop will
>> >>> simply erase or replace the Content-Integrity header value. A secret
>> >>> key
>> >>> unknown to that middleware is required to make the integrity hash
>> >>> break
>> >>> when it tries this.
>> >>>
>> >>> AYJ
>> >>>
>> >>
>> >>
>> >
>> > --
>> > Mark Nottingham
>> > http://www.mnot.net/
>> >
>> >
>> >
>> >
>> >
>>
>

Received on Wednesday, 11 July 2012 18:31:24 UTC