Re: MACing HTTP requests/responses (Re: Content-Integrity header)

On Thu, Jul 12, 2012 at 9:32 AM, Phillip Hallam-Baker <hallam@gmail.com> wrote:
> I suspected as much, but is 're-invent chunked in HTTP/1.1' any more
> practical than 'make chunked worked in HTTP/1.1'.
>

Unfortunately no. For 1.1 it would appear that the only practical
approach would be An Ugly Hack at either the content or
content-encoding level, neither of which are great options for many
obvious reasons.

> This might be something that we just need to make happen in HTTP/2.0
> as part of multiplexing. I am willing to spend some of my personal
> time on that.
>

+1 ... I would very much like to see integrity checking added to
http/2.0 as well as a stronger requirement to support trailers in
general.




> From a Web Services point of view, I think we can manage with an
> integrity header for cases where we need to support intermediaries and
> restrict trailers to cases where we don't. In most cases we are going
> to be layering over TLS in any case.
>
>
> I am not really proposing this header to compete with TLS. The reason
> I want it is to enable a closer binding of the authentication layer to
> the content than I can get in TLS. In particular I observe that:
>
> HTTP + JSON + Integrity Header ~= WS-*
>
>
> I can't run TLS on the type of processor typical in embedded systems
> for all sorts of reasons. But I can get a JSON stack and a frame
> around it.
>
>
> On Thu, Jul 12, 2012 at 12:19 PM, James M Snell <jasnell@gmail.com> wrote:
>> On Wed, Jul 11, 2012 at 6:51 PM, Phillip Hallam-Baker <hallam@gmail.com> wrote:
>>> What are the problems inherent in using Content-Integrity as a Trailer?
>>>
>>> Do the legacy clients botch chunked encoding?
>>>
>>>
>>
>> Well, it appears that many existing 1.1 client, server and
>> intermediary implementations have taken the "trailers are optional"
>> idea to the extreme and either (a) provide no means of appending
>> trailers to the output, (b) provide no means of accessing trailers
>> included in the input or make it quite a bit more difficult to get to
>> them or (c) do not properly forward trailers on as a chunked message
>> flows from hop-to-hop. It's not clear at this point exactly how
>> widespread this problem is but it appears pervasive enough to make a
>> trailer-based content-integrity option problematic at best.
>>
>> - James
>>
>>> On Wed, Jul 11, 2012 at 7:44 PM, James M Snell <jasnell@gmail.com> wrote:
>>>> Phillip, just want to make sure that I'm keeping up with the
>>>> conversation thus far... Because of the problems inherent in using
>>>> Content-Integrity as a Trailer, is the idea then that
>>>> Content-Integrity would be a standard Header and that a new Transfer
>>>> or Content Encoding would be defined that supports an incremental
>>>> integrity check as a component of the encoding?
>>>>
>>>>   GET /some/uri HTTP/1.1
>>>>   Host: example.org
>>>>   TE: integrity
>>>>
>>>> For instance... something like...
>>>>
>>>>   HTTP/1.1 200 OK
>>>>   Content-Type: application/octet-stream
>>>>   Content-Integrity: SHA-256; modifier=123...; param="..."
>>>>   Transfer-Encoding: integrity
>>>>
>>>>   10
>>>>   {chunk of bytes}
>>>>   {digest}
>>>>   10
>>>>   {chunk of bytes}
>>>>   {digest}
>>>>   0
>>>>
>>>> - James
>>>>
>>>> On Wed, Jul 11, 2012 at 4:07 PM, Phillip Hallam-Baker <hallam@gmail.com> wrote:
>>>>> On Wed, Jul 11, 2012 at 2:43 PM, Nico Williams <nico@cryptonector.com> wrote:
>>>>>> I agree that we need something better, and in particular that we ought
>>>>>> to have a MAC instead of a plain hash.  The problem with a MAC is:
>>>>>> whence the key?  Also, what should the MAC be applied to?
>>>>>
>>>>> The MAC should be applied to the 8-bit clean message content (i.e.
>>>>> precisely that which is bounded by Content-Length)
>>>>>
>>>>> If we are talking about Web Services then the key would be established
>>>>> through some application layer key exchange (TBS).
>>>>>
>>>>> The key requirement from a performance standpoint as I see it is that
>>>>> the server has to be able to operate in a stateless fashion which
>>>>> means using a ticket like approach.
>>>>>
>>>>>
>>>>>> Using a MAC, having a shared session key, ties into HTTP
>>>>>> authentication.  We can definitely have a generic MAC for HTTP and say
>>>>>> that HTTP authentication mechanisms that can should output session
>>>>>> keys.  And the HTTP authentication would also have to take care of MAC
>>>>>> algorithm negotiation.  I'd be quite happy with this approach.
>>>>>
>>>>> I think that there is definitely an opportunity to make use of a
>>>>> ticket mode to tie the HTTP authentication to the HTTP channel.
>>>>>
>>>>>
>>>>>> One issue with this approach is: if we always use TLS (but we might
>>>>>> not), why do the extra session crypto?  What do we gain?  Do we need
>>>>>> to worry about content re-writing proxies, say, as in some 3G
>>>>>> networks?  If we always use TLS then it suffices to ensure that a) TLS
>>>>>> provides confidentiality protection, b) the server cert remains the
>>>>>> same for the length of the login session, c) we have a unique,
>>>>>> unpredictable session ID in the headers (what we might call a cookie,
>>>>>> though we don't want it to be a cookie as such).
>>>>>
>>>>> TLS is very large, very complex and was engineered from the assumption
>>>>> that there would be public key credentials on the client. Yes, people
>>>>> can train it to do other tricks, but doing that is a lot more complex
>>>>> than doing what we need in HTTP.
>>>>>
>>>>> In my particular Web Service I am using TLS but still want to have a
>>>>> transport layer authentication protection because I don't want to do a
>>>>> TLS public key negotiation on each transaction and I don't want to be
>>>>> bound to TLS session expiry.
>>>>>
>>>>> In a large commercial environment the TLS processing is often
>>>>> completely offloaded onto an accelerator that strips out the TLS and
>>>>> hands clean IP packets to the Web Service. Another frequent screw case
>>>>> is TLS proxies like bluecoat devices.
>>>>>
>>>>>
>>>>> But even in the simplest TLS use case, the TLS security context is
>>>>> really not exposed to the Web Server or the client in the way you
>>>>> would need to use it for Web Services authentication in the commonly
>>>>> used APIs. The problem is that TLS is designed to conceal all the
>>>>> complexity of crypto from the application. That is why it was called
>>>>> SSL at the start.
>>>>>
>>>>>
>>>>>> In one post you talked about sequencing and replay protection for
>>>>>> chunks.  Adding that to the MAC really gets us close to the MIC token
>>>>>> features/design from RFCs 1964/4121 (Kerberos GSS mech).  We're
>>>>>> talking about having a sequence number.  As you say: this isn't
>>>>>> difficult; we've been doing this for a loooong time in Kerberos land.
>>>>>
>>>>> Heh, you could use a Kerberos token in my Omnibroker protocol if you
>>>>> wanted to. But since it is an opaque string of bytes as far as the
>>>>> client is concerned, well there is no reason to tie it to any one
>>>>> approach.
>>>>>
>>>>> People have been using kerberized cookies for years. The problem being
>>>>> that the cookie is not at all bound to the requests or responses.
>>>>>
>>>>>> Note that there's no need for sequence numbers to randomized given
>>>>>> that we have session keys, but sequence number windows add to the
>>>>>> state to be kept on the server side -- can we tolerate that? Note that
>>>>>> while session key state might be kept on the client in an encrypted
>>>>>> state ticket, session number windows cannot safely be kept that way --
>>>>>> they must be kept locally.  I tend to think that sequencing and replay
>>>>>> protection are the responsibility of the application -- all it needs
>>>>>> to do is add a sequence number to the chunks and manage its own
>>>>>> [per-resource] sequence number windows.
>>>>>
>>>>> The way I was thinking of helping the application was to provide a
>>>>> feature that allows Content-Integrity header to specify a key modifier
>>>>> as well as a ticket. The key used to calculate the MAC would then be
>>>>> the XOR of the modifier and the authentication key associated with the
>>>>> ticket.
>>>>>
>>>>>
>>>>>> Altogether we need: a session key identifier in the headers (this
>>>>>> should imply algorithm selection), a direction identifier (or separate
>>>>>> keys for each direction), a sequence number if we need sequencing
>>>>>> and/or replay detection, what content to MAC, and the MAC itself.
>>>>>
>>>>> Direction is already implicit in HTTP requests/responses.
>>>>>
>>>>> I don't think we need sequencing, we do need a modifier capability though.
>>>>>
>>>>>> Regarding what to MAC: the direction flag (unless we have diff keys
>>>>>> for each direction), the channel binding for the TLS channel (if we
>>>>>> have it), the URL?, some subset of headers? and the body.  Note that
>>>>>> applying the MAC to any headers requires that we say something about
>>>>>> canonicalization (e.g., "use the headers exactly as sent") and
>>>>>> canonical order (if a subset of headers) (e.g., "in the relative order
>>>>>> of appearance").  Header and body content need to be unambiguously
>>>>>> separated in the MAC input.  Obviously we can't MAC all headers: some
>>>>>> might be added by proxies, for example.
>>>>>
>>>>> I don't see the need to MAC any headers for a Web Service application.
>>>>> Put all the information in the content block.
>>>>>
>>>>> Otherwise we would have to do the sort of thing that DKIM does to sign
>>>>> headers and copy them.
>>>>>
>>>>>
>>>>> --
>>>>> Website: http://hallambaker.com/
>>>>>
>>>
>>>
>>>
>>> --
>>> Website: http://hallambaker.com/
>
>
>
> --
> Website: http://hallambaker.com/

Received on Thursday, 12 July 2012 17:40:49 UTC